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A  FACTOR  ANALYSIS  OF  THE  SYMBOLIC-EVALUATION  ABILITIES* 


THE  PROBLEM 


Attempting  to  understand  human  intellectual 
processes,  philosophers  and  psychologists  have  tra¬ 
ditionally  set  the  goal  of  achieving  a  conceptual  sys¬ 
tem  embracing  all  intellectual  activities.  Within  the 
lists  of  intellectual  concepts  are  such  words  as  deci¬ 
sion,  estimation,  evaluation,  hypothesizing,  judgment, 
and  verification.  All  of  these  thought  processes  have 
one  thing  in  common;  they  produce  no  new  informa¬ 
tion;  they  only  oper.ate  upon  cognized  or  produced  in¬ 
formation  and  result  in  a  decision.  They  serve  the 
function  of  stabilizing  an  uncertain  state  of  affairs  or 
of  clarifying  a  somewhat  unstructured  situation. 

The  stabilizing  and  clarifying  functions  of  these 
Judgmental  processes  made  them  amenable  to  early 
experimental  investigation:  subjects  merely  had  to 
choose  between  two  or  more  specified  courses  of  ac¬ 
tion  when  presented  with  a  stimulus  to  be  judged.  The 
results  of  judgment  experiments  were  similar  to  those 
of  the  early  reaction-time  and  perceptual  experiments 
in  one  respect;  the  main  effects  of  the  experimental 
variables  were  often  obscured  by  large  within-group 
response  variability.  Whereas  individual  differences 
very  early  became  a  problem  in  connection  with 
reaction-time  experiments,  andresulted  in  broadened 
investigation  in  the  perceptual  area,  in  the  form  of 
psychophysics,  it  had  little  effect  upon  later  attempts 
to  study  judgmental  processes  empirically. 

The  present  study  is  anattempt  to  approach  em¬ 
pirically  die  probletp  of  describing  judgmental  pro¬ 
cesses  from  the  standpoint  of  individual  differences, 
as  a  step  toward  understanding  those  processes.  It 
attempts  to  Identify  basic  traits  with  respect  to  which 
Individuals  differ  In  judgmental  performances.  Thus, 
for  this  study.  Individual  differences  In  judgmental 
tasks  are  considered  In  terms  of  traits,  more  speci¬ 
fically,  Intellectual  abilities. 

In  1960,  Guilford  and  Merrlfleld  described  a 
theoretical  model  called  the  structure  of  Intellect, 
based  upon  a  number  of  Intellectual  abilities  Identi¬ 
fied  In  past  factor-analytic  studies.  The  model  offers 
a  definition  of  Intelligence  that  is  free  from  the  oper¬ 
ational,  but  circular,  definition  of  intelligence  offered 
by  Boring  (1923)  that  "Intelligence  Is  what  the  tests 
test.  "  In  the  structure-of-lntellect  model,  the  pri¬ 
mary  Intellectual  abilities  are  classified  in  terms  of 
three  dimensions;  operations,  contents,  and  products. 
There  are  five  operations;  cognition,  memory,  diver¬ 
gent  production,  convergent  production,  and  evalua¬ 
tion,  which  the  operator  can  perform  upon  informa¬ 
tion.  There  are  four  kinds  of  contents,  or  categories 
of  Information,  upon  which  the  organism  can  perform 
the  operations.  The  four  contents  are  called  figural. 


*This  study  Is  one  of  a  series  conducted  by  the 
Aptitudes  Research  Project  at  the  Universl^  of 
Southern  California,  under  Contract  Nonr-228(20) 
with  the  Office  of  Naval  Research,  Personnel  and 
Training  Branch.  The  Ideas  expressed  here  are  our 
own  and  In  no  way  reflect  tbs  views  of  that  agency . 
This  material  niay  be  reproduced  for  any  purposes  of 
the  United  States  Government.  Among  the  authors  , 
Guilford  has  served  as  Responsible  Investigator  and 
Director  of  the  Project;  Hoepfner  as  Assistant 
Director  and  Study  Leader;  Merrlfleld  as  Assistant 
Director  during  the  early  stages  of  this  study. 


symbolic,  semantic,  and  behavioral.  The  third  dimen¬ 
sion  of  the  model  represents  the  six  kinds  of  products: 
units,  classes,  relations,  systems,  transformations, 
and  implications.  The  products  are  the  results  of  the 
operator's  psychological  processing  of  information. 
The  intersection  of  a  given  operation,  content,  and 
product  implies  a  unique  factor  of  intelligence.  This 
results  in  the  model's  prediction  of  120  abilities,  61 
of  which  had  received  some  empirical  verification  be¬ 
fore  the  results  of  this  study  became  known. 

This  study  selected  for  investigation  the  six  fac¬ 
tors  of  symbolic  evaluation  predicted  from  the  struc- 
ture-of-intellect  model.  Evaluation  factors  are  con¬ 
cerned  with  abilities  in  which  the  intellectual  process 
is  reaching  decisions  or  making  choices  about  the 
goodness  of  information  in  terms  of  criteria  of  iden¬ 
tity,  consistency,  and  goal  satisfaction.  The  evalua¬ 
tion  abilities  hold  special  importance  to  investigators 
of  thinking  in  general  because  of  the  involvement  of 
those  abilities  in  almost  all  creative  problem  solving, 
particularly  at  the  last  stages  of  problem  solving, 
where  solutions  must  be  accepted,  rejected,  or  slated 
for  revision.  The  symbolic  factors  are  abilities  in 
which  the  organism  operates  upon  information  in  the 
form  of  signs  or  symbols  suchas  letters  and  numbers. 


The  six  factors  selected  for  experimental  in¬ 
vestigation  are;  evaluation  of  symbolic  units  (ESU), 
evaluation  of  symbolic  classes  (ESC),  evaluation  of 
symbolic  relations  (ESR),  evaluation  of  symbolic  sys¬ 
tems  (ESS),  evaluation  of  symbolic  transformations 
(EST),  and  evaluation  of  symbolic  implications  (ESI). 
Although  one  of  the  above  factors,  ESR,  was  pre¬ 
sumed  to  have  been  discovered  (Guilford  and  Hoepfner, 
1963),  Its  existence  as  ESR  was  considered  very  ten¬ 
uous  and  is  therefore  considered  as  an  experimental 
factor  In  this  study  rather  than  a  reference  factor. 

An  objective  of  some  Importance  in  this  study 
is  that  it  represents  another  attempt  to  validate  Gull- 
ford's  model  as  a  source  of  hypotheses  for  defining 
and  Isolating  factors  of  human  intelligence.  If  the 
model  continues  to  generate  concepts  which  can  be 
found  to  represent  unique  abilities,  then  its  contribu¬ 
tion  to  psychological  explanation  and  prediction  will 
be  further  substantiated. 

Not  only  is  the  existence  of  new  factors  deduced 
from  the  model,  but  the  model  further  offers  opera¬ 
tional  specifications  for  the  measures  needed  for  the 
factors.  Stich  measures,  which  serve  as  the  "em¬ 
pirical  world"  for  verification  of  the  factors  and  the 
model,  are  then  available  as  Instruments  for  study  of 
the  traits  in  new  Investigations  involving  those  traits. 
Today's  new  factors  and  their  experimental  tests  be¬ 
come  tomorrow's  reference  concepts  and  marker 
tests  for  use  in  other  kinds  of  investigations.  They 
also  become  available  for  the  applied  psychological 
prediction  and  selection,  which  is  the  ultimate  social- 
value  testing  of  the  model  itself. 

The  potential  value  of  the  symbolic-evaluation 
tests  as  measures  of  constructs  of  intelligence  Is 
considerable  over  a  variety  of  applied  problems  In 
psychology.  Peterson,  et  al.  ( 1963)  reported  that 
five  of  tlM  newly  designed  symbolic  -  evaluation 
factor  tests  contributed  to  the  prediction  of  success 
In  high- school  general  mathematlce  and  algebra. 
Three  factors  ,  ESR  ,  ESS  ,  and  EST  ,  each 
hypothesised  to  be  represented  by  one  test  , 
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contributed  significantly  to  the  predictions.  The  DAT 
subtest,  Clerical,  thought  at  that  time  to  be  a  test  of 
ESU  (and  Ihe  results  of  this  study  support  that  view), 
consistently  contributed  predictive  variance,  second 
only  to  the  Numerical  Ability  subtest,  when  four  DAT 
subtests  were  employed  as  predictors  of  success. 
There  is  considerable  promise,  therefore,  that  meas¬ 
ures  of  symbolic -evaluation  factors  in  general  will 
aid  in  selecting  or  classifying  students  for  mathema¬ 
tics  programs. 

Vinacke  (1952)  stresses  the  value  of  evaluative 
thinking,  especially  in  the  semantic  area,  as  an  im¬ 
portant  component  in  the  final  process  of  creative 
production,  i.  e.  ,  verification.  There  is  no  reason 
not  to  assume  an  equal  value  for  symbolic  evaluative 
thinking  when  the  creative  production  is  symbolic  in 
nature.  The  creative  mathematician  and  (iie  mathe¬ 
matical-theoretical  scientist  must  evaluate  their  nu¬ 
merical  or  symbolic  formulations  in  terms  of  identity, 
consistency,  and  other  criteria  of  goal  satisfaction. 
In  other  words,  each  must  employ  his  symbolic- 
evaluation  abilities  as  he  proceeds,  and  especially  as 
one  of  the  last  steps  in  his  creative  production. 

Guilford  (1964)  goes  even  further  in  describing 
his  model  of  creative  problem  solving  by  stating: 

Another  major  feature  (of  the  model)  is  the 
universal  dependence  of  all  operations  upon 
evaluation.  In  general,  behavior  is  self  regu¬ 
lating  and  self  correcting,  through  the  principle 
of  feedback  information.  An  example  is  seen  in 
the  operation  of  evaluation,  which  helps  to  se¬ 
lect  information  at  the  filtering  stage  near  the 
point  of  input  and  to  reject  or  accept  informa¬ 
tion  in  the  operations  of  cognition  and  produc¬ 
tion.  In  general,  the  operation  of  evaluation 
appears  to  serve  a  filtering  function  as  well  as 
other  functions  (p.  8). 

Even  in  such  current  psychologically  popular 
fields  as  computer  simulation  of  thought  processes, 
more  exact  knowledge  of  such  abilities  will  be  valu¬ 
able.  Attempts  to  program  certain  mental  processes, 
particularly  with  regard  to  symbolic  content,  have 
repeatedly  revealed  the  lack  of  information  av'ailable 
on  how  human  beings  carry  out  seemingly  simple 
thought  processes.  Such  frustrated  simulation  re¬ 
search  has  led  to  the  return  to  the  study  of  human 
thought  processes.  The  knowledge  that  is  gained 
concerning  the  processes  involved  in  the  symbolic- 
evaluation  abilities  could  very  likely  be  useful  in 
simulation  research,  which  results,  in  turn,  in  great¬ 
er  knowledge  of  the  thought  processes  involved. 


REVIEW  OF  THE  LITERATURE 
Some  General  Sources 

The  philosophical  literature  is  well-sprinkled 
with  discourses  on  evaluation,  choice,  and  decision 
making.  In  all  cases,  however,  the  major  concern 
is  with  individual  and  cultural  values  and  morality, 
and  the  experiential  development  of  values.  Empiri¬ 
cal  investigation  of  such  concepts  seems  to  be  wholly 
lacking.  These  concepts  would  need  a  great  deal  of 
specific  research  before  they  could  be  employed  as 
variables  in  a  study  like  the  present  one. 

Reports  of  educational  research  are  primarily 
concerned  with  critical  thinking,  which  appears  to  be 
limited  to  the  evaluation  of  conceptual  information  in 
terms  of  logical  criteria.  Tha  educator  is  interested 
in  learning  how  to  impart  to  his  students  the  ability  to 
think  critically.  Although  such  research  is  well 


worthwhile,  it  seems  to  be  premature;  the  educator 
can  hardly  expect  to  teach  critical-thinking  skills 
when  nothing  but  general  and  vague  notions  are  held 
about  their  nature. 

The  psychological  literature  also  abounds  with 
reports  on  research  bearing  upon  judgmental  abilities. 
Depending  upon  the  specific  area  of  such  research, 
different  variables  receive  varying  amounts  of  atten¬ 
tion.  In  psychophysics,  the  major  concern  is  with  the 
relations  between  the  stimuli  and  the  subjects' .judg¬ 
ments  of  them.  The  task  in  these  psychophysical  ex¬ 
periments  is  to  find  the  lawful  relationships  existing 
between  the  continue  of  stimulus  dimensions  and  the 
continue  of  judgment. 

Attempting  to  develop  a  theory  of  judgment  and 
choice,  Restle  (1961)  states: 

A  cognitive  position  on  judgment  argues  that 
judgment  depends  both  oh  external  events  (stim¬ 
ulus)  and  the  observer,  taken  together  as  an  in¬ 
teracting  system  (p.  189). 

This  position  is  clearly  similar  to  that  taken  by  the 
psychophysicists,  but  Restle  represents  part  of  a-new 
development  in  the  study  of  judgment— applications 
and  explanations  of  judgment,  choice,  and  decision 
through  mathematical  models.  The  probabilistic 
mathematical  models  have  been  applied  to  the  inves¬ 
tigation  of  decision  processes  under  the  subject  areas 
of  utility  theory,  information  theory,  decision  theory, 
and  game  theory. 

The  present  study  is  directly  concerned  with  the 
very  individual  differences  that  the  above-mentioned 
methods  of  investigation  have  ignored.  The  approach 
through  individual  differences  is  needed  in  order  to 
derive  the  concepts  descriptive  of  psychological  events 
in  decision  making,  making  possible  something  more 
than  a  superficial  approach  to  theory  construction. In¬ 
dividual  differences  are  generally  investigated  by 
means  of  mental  tests  and,  currently,  by  factor  anal¬ 
ysis  or  some  related  multivariate  procedure.  For 
this  reason,  the  review  of  the  literature  here  will  fo¬ 
cus  upon  previous  factor-analytic  research. 


Factor -Analytic  Studies 

Over  the  years,  only  by  virtue  of  small,  inci¬ 
dental  steps  was  attention  directed  to  the  evaluation 
area  by  virtue  of  factor -analytic  results.  The  first 
of  these  steps  can  be  attributed  to  L.  L.  Thurstone 
(193Ba)  when  he  discovered  the  factor  that  became 
known  as  "perceptual  speed,  "  and  that  later  was  rec¬ 
ognized  as  an  evaluation  factor— the  evaluation  of 
figural  units  (EFU).  The  factor  has  been  most  con¬ 
sistently  defined  by  tests  that  require  the  rapid  com¬ 
parison  of  figural  objects  with  judgments  of  Identity 
versus  non-identity  (Guilford  and  Lacey,  1947).  But 
there  have  been  times  when  there  was  uncertainty  as 
to  whether  it  also  applied  to  tests  requiring  the  iden¬ 
tification  and  matching  of  letters,  numbers,  and  words 
(Thurstone,  1938b;  Coombs,  1941). 

Thus,  the  perceptual-speed  factor  is  of  double 
interest  in  the  present  investigation,  since  the  struc- 
ture-of-intellect  (SI)  model  forecasts  a  distinct  but 
parallel  ability  concerned  with  the  identity  versus 
non-identity  of  literal  material,  and  tests  of  EFU 
could  serve  as  a  model  for  the  hypothesised  ESU  fac¬ 
tor,  which  heretofore  had  not  been  clearly  dennonstra- 
ted.  One  or  two  studies  cited  by  French  (19S1)  give 
some  hope  of  such  differentiation,  e.g. ,  Bechtoldt 
(1947),  but  there  was  nothing  decisive. 
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Another  step  occurred  in  the  Army  Air  Force 
Aviation  Psychology  Program,  when  a  factor  called 
"judgment”  was  found  and  verified  (Guilford  and  Lacey, 
1947).  In  earlier  studies  in  the  Aptitudes  Research 
Project,  however,  such  a  factor  did  not  at  all  consis¬ 
tently  appear  (Nihira,  et  al.  ,  1964). 

A  factor  that  was  found  repeatedly  in  earlier 
Project  analyses  was  first  called  "logical  evaluation," 
most  consistently  brought  out  by  tests  of  the  syllogis¬ 
tic  type.  Since  the  tests  were  of  multiple -choice 
form,  alternative  conclusions  being  given,  not  pro¬ 
duced  by  the  examinee,  the  interpretation  was  an 
evaluative  ability  rather  than  a  deductive  ability. 
There  was  a  seemingly  parallel- factor,,  identified  most 
consistently  by  a  test  called  Symbol  Manipulation, 
which  was  essentially  syllogistic  in  form  but  with  let- 
ter  symbols  rather  than  words.  This  is  the  nearest 
that  the  Project  has  come  to  demonstrating  previously 
a  symbolic-evaluation  factor. 

The  Project's  two  previous  investigations  aimed 
specifically  at  evaluation  abilities  concentrated  very 
much  on  the  semantic  factors.  Hertzka,  et  al.  ,  (1954) 
confirmed  the  logical-evaluation  factor,  which  was 
later  interpreted  as  the  EMR  factor  in  the  SI  model. 
They  also  found  a  factor  called  "speed  of  evaluation," 
for  the  rather  incidental  (as  it  turned  out)  reason  that 
it  was  a  feature  of  highly  speeded  tests.  A  third 
evaluation  factor  was  called  "experiential  evaluation," 
because  it  was  distinguished  by  tests  that  called  for 
judgments  based  on  consistency  with  past  experience. 
This  factor  was  later  placed  in  the  cell  for  EMS  in  the 
SI  model,  since  the  consistency  involved  pertained  to 
comparisons  of  parts  of  situations,  which  could  well 
be  regarded  as  systems. 


In  a  recently  completed  Project  investigation, 
the  analysis  was  aimed  at  the  six  semantic-evaluation 
factors  forecast  by  the  SI  model  (Nihira,  et  al. ,  1964). 
The  factor  formerly  called  "speed  of  evaluation"  was 
apparently  verified,  but  it  was  defined  by  tests  de¬ 
signed  for  factorEMU,  hence  was  defined  as  the  abil¬ 
ity  to  evaluate  semantic  units.  A  new  factor  was 
found  to  represent  the  cell  for  EMC  and  one  for  the 
cell  for  EMR,  and  factor  EMS  was  confirmed.  The 
logical-evaluation  factor,  previously  identified  with 
the  cell  for  EMR,  was  found  to  be  better  interpreted 
as  factor  EMI,  The  identification  of  five  of  the  six 
semantic -evaluation  factors  lent  much  hope  that  a 
differentiation  of  six  symbolic-evaluation  abilities 
might  be  effected.  There  was  also  the  possibility  that 
to  demonstrate  a  certain  symbolic-evaluation  factor, 
tests  modeled  after  those  for  the  parallel  semantic 
factor  might  serve.  Advantage  was  taken  of  such 
parallels  in  some  instances,  although  the  semantic- 
evaluation  study  was  not  completed  before  the  sym¬ 
bolic-evaluation  study  was  very  well  along,  so  that 
in  most  instances  the  model  tests  had  not  been  estab¬ 
lished  as  good  tests  for  their  respective  factors. 


It  is  of  interest  to  note  that  in  a  study  designed 
to  learn  more  about  the  intellectual  abilities  pertinent 
to  success  in  mathematics,  Canisia  (1962)  employed 
quite  a  number  of  symbolic  tests,  some  of  which  ap¬ 
pear  to  have  some  evaluation  variance.  Examples 
are:  Number  Series  2  (which  would  seem  to  be  a 
measure  of  factor  EsS);  Condition*  2  (factor  ESR);  and 
CJualitative  Relationships  (factor  ITSl).  Having  only 
one  representative  test  for  each  factor,  however,  one 
could  not  expect  these  three  factors  to  emerge.  But 
it  would  appear  that  Canisia  was  working  in  the  right 
directions. 


THE  HYPOTHESIS 

The  formal  hypothesis  is  concerned  with  the 
demonstration  of  six  selected  intellectual  abilities 
predicted  from  the  SI  model,  using  the  experimental 
tests  designed  to  demonstrate  them.  Guilford  (1960) 
has  concisely  stated  the  requirements  of  a  good  factor- 
analytic  experiment  whose  objective  is  to  demonstrate 
something  of  basic  psychological  significance.  First, 
the  investigator  makes  hypotheses,  basedupon  reason¬ 
able  inferences  from  previous  research  or  upon  im¬ 
plications  from  a  model  or  theory,  according  to  which 
individuals  are  expected  to  differ  from  one  another. 
Such  hypotheses  are  given  attention  in  the  first  part 
of  this  section.  Second,  the  investigator  develops 
measures  that  will  broadly  sample  behavior  indicat¬ 
ing  the  individual  differences.  The  second  part  of 
this  section  is  concerned  with  the  measures  developed 
to  test  the  hypothesis. 

Statement  and  Elaboration  of  the  Hypothesis 

The  major  hypothesis  is  that,  in  the  population 
employed  in  this  study,  six  separate  factors  bf  sym¬ 
bolic  evaluation,  predicted  by  the  SI  model,  exist, 
distinct  from  one  another  and  from  factors  represen¬ 
ted  in  certain  other  areas  of  the  model.  Furthermore, 
these  six  factors  are  not  restricted  to  any  one  kind  of 
symbol.  In  other  words,  tests  defining  any  one  of  the 
six  factors  could  be  composed  of  words,  numbers,  or 
letters.  Moreover,  each  of  the  six  factors  is  defined 
by  tests  involving  both  sensitivity  and  estimation,  con¬ 
cepts  to  be  explained  later. 

The  formal  hypothesis  is  somewhat  complex,  an 
attribute  not  conducive  to  unambiguous  evaluation  of 
results.  It  is  necessary  to  look  more  closely  at  some 
of  the  specific  aspects  of  the  hypothesis.  If  one  de¬ 
fines  the  object  of  this  study  as  the  evaluation  of  sym- 
bolic  products,  careful  consideration  must  be  given 
to  each  of  the  underscored  words,  in  order  to  clarify 
all  aspects  of  the  formal  hypothesis. 

A  Definition  of  Evaluation 


In  formulating  a  definition  of  "evaluation"  for 
this  study,  several  alternatives  were  considered.  We 
shall  mention  two  of  them,  since  they  have  a  direct 
bearing  on  the  distinction  between  "sensitivity"  tests 
and  "estimation"  tests,  and  then  state  a  broader  defi¬ 
nition  that  embraces  both  conceptions. 

In  a  definition  that  equates  evaluation  to  "sensi¬ 
tivity  to  error,  "  the  term  "error"  is  interpreted 
broadly  to  include  any  kinds  of  defects,  deficiencies, 
departures,  inconsistencies,  incongruities,  etc.  This 
view  implies  absolute  judgments;  a  thing  is  perfectly 
all  right  or  is  identical  with  another,  or  it  is  not. 
Some  individuals  can  detect  such  "errors"  where 
others  cannot.  Those  who  can  are  more  able  with 
respect  to  evaluative  abilities  than  those  who  cannot. 
This  is  not  to  say  that  there  is  a  dichotomy  of  indivi¬ 
duals;  they  can  still  vary  by  small  degrees  along  a 
continuum  of  greater  or  less  sensitivity. 

In  the  definition  that  emphasizes  "estimation," 
it  is  implied  that  individuals  also  make  relative  judg¬ 
ments.  When  items  of  information  fall  short  or  devi¬ 
ate  from  a  standard,  one  may  deviate  farther  than 
another.  It  may  be  obvious  that  all  the  items  of  in¬ 
formation  depart  from  the  standard,  but  which  one 
deviates  ieast?  Where  sensitivity  tests  typically  call 
for  absolute  judgments  of  a  yes-no,  disjunctive  type. 
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estimation  tests  typically  offer  alternative  items  of 
information  and  ask  which  one  deviates  least,  or 
sometimes  (but  rarely)  which  one  deviates  most.  A 
ranking  of  alternatives  is  implied.  This  view  of  esti¬ 
mation  is  concordant  with  that  offered  by  D.  M.  Johnson 
(1955)  when  he  says, 

When  judgments  are  expressed  on  a  continu¬ 
ous  scale,  .  ,  .  the  term  estimating  may  be 
used,  as  a  special  case  of  judgment.  Because 
the  scales  used  in  estimation  are  continuous, 
there  are  not  boundaries  between  the  categories 
.  .  .  (p.  284). 

Actually,  the  two  views  can  be  brought  logically 
under  the  same  definition  of  evaluation.  In  both  cases, 
a  standard  of  some  kind  is  implied.  In  both  cases, 
criteria  for  judgment  are  implied.  A  definition  em¬ 
bracing  both  views  would  read:  "Evaluation  is  a  mat¬ 
ter  of  decision  concerning  criterion  satisfaction.  " 
This  is  the  definition  adopted  as  the  basis  for  plan¬ 
ning  this  study.  One  of  the  objectives  of  this  study 
was  to  determine  whether  tests  embodying  the  sensi¬ 
tivity  principle  and  those  embodying  the  estimation 
principle  would  both  Indicate  the  same  kinds  of  ability, 
thus  justifying  subsuming  them  both  under  a  single 
definition  of  evaluation. 


Criteria  for  Evaluation.  When  it  is  said  that 
evaluation  is  concerned  with  criterion  satisfaction,  it 
is  then  necessary  to  give  attention  to  what  kinds  of 
criteria  are  suitable  for  use  in  tests  of  evaluation 
abilities.  Some  of  the  traditional  criteria  have  been: 
identity  vs.  deviation,  completeness  vs.  incomplete¬ 
ness,  compatibility  vs.  incompatibility,  congruity  vs. 
incongruity,  effectiveness  vs.  ineffectiveness,  and 
suitability  vs.  unsuitability.  For  testing  purposes,  a 
criterion  must  be  of  a  type  that  can  be  communicated 
to  the  examinee.  As  will  be  seen  in  discussions  of 
the  results,  some  additional  kinds  of  criteria  were 
included  in  this  study,  such  as  popularity  vs.  unpopu¬ 
larity  (frequency  of  usage)  and  highly  probable  vs. 
improbable.  With  symbolic  material  to  be  evaluated, 
there  was  no  question  about  using  either  esthetic  or 
moral  criteria.  Questions  regarding  those  two  kinds 
of  criteria  will  eventually  arise  in  analyses  pertaining 
to  figural  information  on  the  one  hand  and  behavioral 
information  on  the  other. 


Definition  of  Symbolic  Information 

The  whole  logic  of  symbolic  communication  as 
compared  with  conceptual  or  semantic  communication 
is  that  more  precision  can  be  had  due  to  the  denotative 
inflexibility  of  symbols.  One  might  then  ask  the 
question,  "What  is  there  to  evaluate  in  connection  with 
symbolic  information?"  We  might  expect  some  dif¬ 
ferent  aspects  to  evaluation  of  symbolic  information 
than  those  that  apply  to  semantic  information,  which 
is  relatively  rich  with  connotative  meaning. 

Several  different  varieties  of  information  con¬ 
form  to  the  definition  of  symbols  stated  by  Guilford 
and  Hoepfner  (1963):  "Informa'tion  in  the  form  of  signs, 
having  no  significance  in  and  of  themselves.  "  (p.  2). 

The  clearest  example  of  a  symbol  is  a  number.  Num¬ 
bers  have  no  significance  in  and  of  themselves,  yet 
can  be  evaluated  for  numerical  identity,  order,  or 
consistency,  with  respect  to  other  numbers.  Letters 
also  conform  to  the  definition  when  they  are  processed 
in  terms  of  their  literal  properties  rather  than  their 
figural  properties.  Syllables  can  be  symbolic  units, 
as  v/ell  as  words,  when  their  semantic  meanings  are 
not  relevant  to  the  task,  as  in  breaking  words  into 


syllables  or  in  word  compounding.  All  these  types  of 
symbols  were  used  in  various  experimental  tests  in 
this  study. 


Definition  of  Products 

The  six  kinds  of  products  of  the  SI  model  have 
been  defined  in  a  number  of  places,  but  will  be  very 
succinctly  defined  here.  Units  are  segregated  items 
of  information  having  thing  character.  Classes  are 
groups  of  items  of  information  having  common  pro¬ 
perties.  Relations  are  meaningful  connections  be¬ 
tween  units.  Systems  are  organized  or  structured 
complexes  of  units  and  relations.  Transformations 
are  changes  or  redefinitions  of  known  items  of  infor¬ 
mation.  Implications  are  in  the  form  of  expectancies, 
predictions,  or  consequences  of  information. 

This  aspect  of  the  hypothesis,  the  existence  of 
six  distinct  product  factors  of  symbolic  evaluation  , 
provides  the  major  problem  of  this  study.  The  demon¬ 
stration  of  these  six  factors,  or  the  failure  to  do  so, 
would  confirm  or  fail  to  confirm  the  model  from  which 
the  hypothesis  was  deduced.  The  tests  designed  as 
measures  for  each  of  the  products  will  be  described 
in  detail  later. 


Varieties  of  Experimental  Tests 

With  three  kinds  of  symbols  available  and  with 
the  distinction  between  sensitivity  and  ’•stimation 
tests,  for  a  completely  systematic  experimental  de¬ 
sign  it  would  have  been  desirable  to  have  six  experi¬ 
mental  tests  for  every  product  factor.  No  effort  was 
made  to  achieve  fully  this  kind  of  coverage  with  ex¬ 
perimental  tests,  and  it  was  difficult  to  achieve  all 
six  kinds  of  tests  with  each  product.  Table  1  shows 
how  far  the  test  development  went  in  the  direction  of 
complete,  systematic  coverage. 

The  two  major  columns  represent  sensitivity  vs. 
estimation  types  of  tests;  those  involving  absolute 
judgments  vs.  those  requiring  relative  judgments. 
Comparing  the  two  columns  it  will  be  seen  that  the 
balance  is  fair.  For  particular  factors  the  balance  is 
good  in  the  case  of  three,  but  not  balanced  in  the  other 
three.  Overall  factors  there  is  a  fair  balance  between 
the  three  kinds  of  symbols,  with  a  few  tests  being 
composed  of  combinations  of  two  or  three  kinds. 
The  re  proved  to  be  enough  dispersion  of  the  cond  itions 
to  make  possible  answers  as  to  whether  both  sensi¬ 
tivity  tests  and  estimation  tests  serve  to  measure 
these  evaluation  factors  and  whether  the  kind  of  sym¬ 
bol  mtdtes  a  difference  in  the  success  of  tests. 


Descriptions  of  Factor  Measures 


Reference  Factors 

An  important  goal  for  the  well-designed  factor- 
analytic  test  of  factor  hypotheses  is  not  only  to  deter¬ 
mine  what  the  experimental  constructs  are  but  also 
what  they  are  not.  The  formal  hypothesis  states  that 
the  six  factors  of  symbolic  evaluation  are  not  only 
distinct  from  one  another,  but  are  also  distinct  from 
other  factors  deduced  from  the  model.  For  this  rea¬ 
son,  a  number  of  marker  tests,  known  from  previous 
experience  to  measure  reference  factors,  were  in¬ 
cluded  in  the  analysis  to  demonstrate  the  distinctness 
of  the  new  experimental  factors  from  factors  already 
known, 
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Table  1 

Semi -Balanced  Factorlal-LUce  Design  of  25  New  Symbolic-Evaluation  Tests 


Estimation 


ComblMtlon* 


S  Test 


Symbol  Reasoning 


The  usual  test  of  the  distinctness  of  experimental 
factors  is  made  by  selecting  for  s  imultaneous  analysis 
reference  factors  thatmight  possiblybe  identical  with 
the  experimental  factors.  Of  all  the  non-evaluation 
factors,  those  of  cognition  were  suspected  of  being 
most  likely  to  be  contused  with  the  experimental  eval¬ 
uation  factors.  One  reason  is  that  it  takes  care  to 
construct  an  evaluation  test  so  that  it  does  not  offer 
necessary  cognition  problems  of  sufficient  difficulty 
to  introduce  some  cognition  variance  into  the  total 
scores,  or,  indeed,  that  it  does  not  become  a  cogni¬ 
tion  test  instead  of  an  evaluation  test.  For  . this  rea¬ 
son,  tests  of  five  parallel  symbolic-cognition  factors 
were  selected  for  inclusion  in  the  battery. 


A  measure  of  numerical  facility,  recognized  as 
a  memory  factor  in  the  SI  model,  was  included  to  de¬ 
fine  that  factor.  The  reason  was  the  large  proportion 
of  number  tests  in  the  battery,  some  of  which  might 
have  some  numerical-facility  variance  that  should  be 
segregated. 

Because  scores  on  one  of  the  classes -evaluation 
tests  (Best  Number  Class),  designed  for  the  factor 
ESC,  were  expected  to  depend  upon  ability  to  produce 
rapidly  responses  to  be  evaluated,  two  measures  of 
the  parallel  divergent-production  factor,  DSC,  were 
employed  as  marker  tests.  In  addition,  this  analysis 
afforded  an  opportunity  to  learn  more  about  the  nature 
of  the  DSC  factor,  which  had  not  previously  been  very 
clearly  demonstrated. 

It  might  be  expected  that  most  convergent- 
production  tests  would  involve  some  evaluative  acti¬ 
vity,  since  competing  answers  to  items  must  be  re¬ 
jected  in  favor  of  the  one  right  answer,  hence  such 
tests  might  have  some  evaluation  variances.  Three 
of  the  ell -known  symbolic  convergent-production 
factors  were  represented  by  two  marker  tests  each. 

To  determine  clearly  the  separateness  of  the 
perceptual-speed  factor  EFU  from  its  parallel  sym¬ 
bolic  counterpart  (ESU),  two  marker  tests  for  EFU 
were  included.  This  was  expected  to  clear  up  the 
ambiguity  that  had  been  left  with  regard  to  these  two 
factors  in  previous  investigations,  as  discussed 
earlier. 

Although  not  considered  as  a  reference  factor, 
an  anticipated  motivational  variance  seemed  to  need 
accounting  for.  It  was  expected  that  symbolic  tests 
might  have  systematically  different  degrees  of  appeal 
for  individuals,  which  might  contribute  to  the  appear¬ 
ance  of  a  general  evaluation  factor  or  a  substantial 
appearance  of  obliqueness  among  the  evaluation  fac¬ 
tors.  In  order  to  segregate  such  sources  of  variance, 
four  experimental  measures  were  included  in  an  at¬ 
tempt  to  isolate  a  possible  motivation  factor. 


The  Marker  Tests 

In  the  following  paragraphs,  each  of  the  13  ref¬ 
erence  factors  and  their  marker  tests  is  mentioned  in 
turn.  Descriptions  of  the  tests,  with  sanqjle  items, 
will  be  found  in  the  Appendix  of  this  Report.  Most  of 
the  symbolic  marker  tests  had  been  used  in  a  pre¬ 
vious  analysis  (Guilford,  et  al.  ,  1961).  Test  data 
from  the  testing  in  that  analysis  were  used  in  item 
analyses  that  were  aimed  toward  shortening  the  tests 
by  eliminating  items  that  (1)  did  not  correlate  well 
with  total  scores  of  their  own  factors  and  (2)  corre¬ 
lated  with  total  scores  of  other  factors.  In  addition 
to  saving  testing  time,  this  procedure  was  expected 
to  make  the  total  scores  more  nearly  univocal  for 
their  own  respective  factors. 


CSU  Tests 

Disemvowelled  Words - E  recognizes  words 

with  vowels  omitted. 

Word  Combinations - E  sees  what  word  can  be 

made  out  of  the  end  of  one  word  and  the  beginning  of 
the  next. 

CSC  Tests 

Number  Classification — ^E  selects  a  number 
that  goes  in  a  class  with  three  other  numbers. 

Number  Group  Naming - E  states  how  three 

numbers  are  alike. 

CSR  Tests 

Seeing  Trends  II— —E  identifies  and  names  the 
trend  in  a  series  of  words,  based  upon  alphabetical 
principles. 

Word  Relations —An  analogies  test  in  which 
spelling  relations  between  words  must  be  seen. 

CSS  Tests 

Circle  Reasoning - E  sees  a  principle  of  order 

in  series  of  circles  and  dashes. 

Letter  Triangle - E  sees  the  alphabetical  prin¬ 

ciple  by  which  letters  are  arranged  in  a  triangle. 

CSI  Tests 

Symbol  Grouping - E  rearranges  a  line  of 

scrambled  X's,  O's,  and  -'s  in  systematic  order, 
making  as  few  moves  as  possible,  involving  fore¬ 
sight. 

Word  Patterns— E  .arranges  words  in  cross¬ 
word -puz^TeTainuoir,  as  efficiently  as  possible. 

CMU  Tests 

General  Vocabulary^— Test  8  of  the  Iowa  Tests 
of  Edu'cational  Development  (ITED)  (Lindquist,  1959). 

Verbal  score  of  the  Preliminary  Scholastic  Ap¬ 
titude  Test  (PSAT)  (ETS,  l959). 

Verbal  score  of  the  Cooperative  School  and  Col- 
lege  Ability  Test  (SCAT)  (ETS,  1958). 

MSI  Tests 

Numerical  Operations— Part  III  of  the  Guilford- 
Zimmeirman  Aptitude  Survey  (1956). 

DSC  Tests 

Number  Grouping— Givena  set  of  several  num¬ 
bers,  E  arranges  and  rearranges  them  in  different 
classes. 

Varied  Symbols - E  classifies  letter  sets  in 

different  ways. 

NSS  Tests 

Operations  Sei^uence— E  states  the  correct 
order  in  which  a  series  of  numerical  operations  must 
be  performed. 

Word  Changes— —E  gives  the  order  in  wdtich 
short  words  must  be  changed,  one  letter  at  a  time,  to 
go  from  starting  word  to  goal  word. 
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NST  Tests 

Camouflaged  Words - E  breaks  up  consecutive 

words  to  find  names  o(  sports  or  games. 

Word  Transformations - E  regroups  a  series 

of  words  in  a  phrase  to  obtain  a  new  phrase. 

NSI  Tests 

Form  Reasoning— E  solves  equations  com¬ 
posed 'orTTguTeTsyrnHoT  combinations. 

Sign  Changes - E  solves  simple  numerical 

equations,  following  rules  about  interchanging  signs. 

EFU  Tests 

Identical  Forms - E  selects  figures  identical 

with  a  standard. 

Perceptual  Speed - Part  IV  of  the  Guilford- 

Zimme'rman  Aptitude  ffurvey  (1956). 


Experimental  Factors 

The  tests  for  the  hypothesized  factors  were  de¬ 
veloped  us  ing  either  of  two  approaches  or  a  combina¬ 
tion  of  the  two.  ^  In  the  first  approach,  specific 
examples  of  tasks  are  deduced  from  the  operation- 
content-product  combination  being  investigated.  For 
example,  the  ability  in  the  cell  EST,  evaluation  of 
symbolic  transformations,  involves  evaluation  of 
changes  in  symbolic  materials.  A  code  can  be  an 
example  of  a  symbolic  change,  and  so  the  test  Decod¬ 
ing  was  developed. 

The  second  approach  emphasizes  tasks  similar 
to  those  for  established  factors  having  one  or  two 
attributes  in  common  with  the  new  factors.  For 
example,  ESU,  evaluation  of  symbolic  units,  and  EFU, 
evaluation  of  flgural  units,  differ  only  with  respect  to 
the  content  category!  test  formats  might  be  very  sim¬ 
ilar.  Four  of  the  25  symbolic-evaluation  tests  were 
revised  from  older  tests.  They  are  Letter  "U",  Sign 
Changes  II,  Symbol  Manipulation,  and  Sound  Grouping. 
The  remaining  21  tests,  developed  by  the  first  and 
second  approaches,  were  written  by  the  first  author. 

During  the  period  of  September,  1961,  through 
March,  1963,  nearly  30  different  pretest  booklets 
were  administered  to  classes  in  psychology  at  the 
University  of  Southern  California,  Los  Angeles  City 
College,  and  Pasadena  City  College.  ^  These  pre¬ 
testings  were  designed  to  obtain  technical  Information 
such  as  the  appropriate  level  of  item  difficulties, 
comprehension  level  of  the  test  instructions,  test 
reliabilities,  and  optimal  time  requirements  for  new¬ 
ly  developed  tests.  Extensive  item  analyses  were 
conducted  whenever  pretesting  information  revealed 
low  reliability  estimates. 

From  the  reliability  and  intercorrelation  data 
obtained  from  pretesting,  25  tests  were  selected  from 
a  pool  of  over  40  tests  especially  designed  or  adapted 
to  measure  the  six  experimental  factors.  The  selected 
tests  had  pretest  reliabilities  in  the  70's  and  .  80's. 

2Much  credit  is  due  Miss  Carolyn  Frame,  who 
assisted  in  the  construction  and  preliminary  analysis 
of  the  experimental  tests. 

Vor  much  of  the  pretestings  we  are  indebted  to 
Dr.  Constance  D.  Lovell  of  the  Department  of  Psy¬ 
chology,  U.  S.  C. ,  Mr.  R.  Jones,  U.S.  C.  Testing 
Bureau,  and  Mr.  H.  Petersen,  Pasadena  City  College. 


Within-factor  intercorrelations  were  generally  con¬ 
siderably  higher  than  between-factor  intercorrela¬ 
tions,  further  ensuring  the  demonstration  of  the  dis¬ 
tinctness  of  the  expected  factors.  The  criteria  of 
high  reliability  and  desirable  correlational  pattern, 
in  conjunction  with  the  considerations  listed  in  Table 
1,  determined  which  tests  were  finally  selected  to 
represent  the  experimental  factors  in  the  final  analy¬ 
sis.  In  the  following  paragraphs,  the  six  experimen¬ 
tal  factors  and  the  tests  selected  to  measure  them  are 
discussed  in  detail. 

ESU  Tests 

The  five  experimental  tests  developed  to  mea¬ 
sure  evaluation  of  the  product  of  units  had  in  common 
symbolic  stimuli  that  are  processed  as  wholes,  rather 
than  separated,  analyzed,  or  classed.  Although  sim¬ 
ilar  symbolic  stimuli  are  employed  in.  tests  of  the 
other  intellectual  products,  the  mental  process  per¬ 
formed  upon  units  must  maintain  the  thing  quality  of 
the  stimuli.  Previous  tests  that  have  aided  in  defin¬ 
ing  factors  of  symbolic  units  involved  the  following 
kinds  of  stimuli;  anagrams,  words,  scrambled  words, 
and  incomplete  words  (CSU);  digits  and  letters  (MSU); 
specified  classes  of  words  in  terms  of  given  initial 
letters,  suffixes,  and  prefixes  (DSU).  Guilford  and 
Hoepfner  (1963)  suggest  tests  employing  letters  and 
digits  as  stimuli  for  measures  of  ESU  based  on  tenu¬ 
ous  prior  studies  (French,  1951).  Construction  of  the 
experimental  ESU  tests  was  based  upon  the  history  of 
parallel  tests  of  symbolic  units  and  the  try-out  of  new 
kinds  of  stimuli. 

The  test  Correct  Spelling  employed  complete, 
common  English  words  as  symbolic  stimuli.  The 
words  function  as  symbols  because  E  is  to  direct 
evaluation  toward  spelling  rather  than  meaning.  E  is 
tested  on  his  sensitivity  to  the  correctness  or  incor¬ 
rectness  of  the  spelled  symbolic  unit.  In  this  case, 
sensitivity  to  spelling  is  based  largely  upon  the  long¬ 
term  retention  of  the  correct  symbolic  elements  of 
standard  English  words.  The  words  employed  as 
items  were  selected  from  lists  of  commonly  mis¬ 
spelled  words  published  in  English  handbooks  and 
secretarial  manuals.  The  test  was  designed  as  a 
power  test,  there  being  enough  time  to  judge  all  spell¬ 
ings.  Tests  very  similar  to  Correct  Spelling  were 
found  by  Thurstone  to  be  loaded  on  the  word-Uuency 
factor  (1938a)  and  on  a  verbal  factor  (1940). 

Derived  from  the  format  of  the  test  Anagrams, 
used  by  Thurstone  (1938a),  Derivations  also  employs 
complete  English  words  as  test  stimuli.  Whereas 
Thurstone 's  test  has  Es  make  as  many  short  words 
as  they  can  in  a  limited  time  from  the  letters  in  a 
large  given  word.  Derivations  supplies  not  only  the 
given  word,  but  also  50  short  words  derived  from  it. 
E's  judgments  are  based  upon  sensitivity  to  the  errors 
in  some  words  that  could  not  be  derived  from  the  long 
given  words.  The  task  of  this  test,  detecting  changes 
in  anagrammatically-derived  words,  naturally  sug¬ 
gests  that  the  product  to  be  evaluated  might  be  trans¬ 
formations.  Although  Derivations  was  originally 
written  as  an  EST  test  and  there  may  be  some  EST 
variance  involved  in  its  scores,  pretesting  led  the 
authors  to  hypothesize  that  the  scores  would  be  more 
heavily  determined  by  ESU  variance.  The  reason 
probably  is  that  it  is  the  end  product  of  a  change  that 
is  to  be  evaluated,  not  the  change,'  as  such.  A  simi¬ 
lar  problem  was  encountered  in  constructing  teats  for 
EMT  (Nihira,  et  al.  ,  1964). 

The  given  stimulus  words  were  selected  on  the 
basis  of  the  number  of  different  letters  they  contain; 
PARENTHESIS,  REPUTATION,  and  COMBINATION. 
Errors  of  some  derived  words  involve  the  use  of  let¬ 
ters  not  in  the  given  word  or  the  use  of  a  letter  more 


often  than  It  appeared  in  the  given  word.  Derivation* 
wa*  designed  a*  a  speed  test.  The  instructions  urge 
the  Es  to  make  their  judgments  rapidly  as  there  would 
be  insufficient  time  for  careful  consideration  of  every 
derived  word. 

Familiar  Letter  Combinations  is  an  experimen¬ 
tal  test  that  has  a  completely  new  type  of  symbolic 
stimuli;  three-letter  syllables.  E  is  to  estimate 
which  of  two  given  syllables  is  more  common  as  a 
part  of  real  English  words.  Familiarity  is  the  cri¬ 
terion  for  decision.  Neither  the  syllables  nor  the 
criter  ia  of  real  words  are  to  be  considered  semantic- 
ally;  only  the  relative  frequencies  of  occurrence  are 
relevant.  The  key  for  this  test  was  determined  from 
the  empirical  frequency  counts  reported  by  Underwood 
and  Schulz  (I960).  The  nonsense  syllables  are  paired 
so  that  the  keyed  syllable  is  far  more  commonly  used 
than  its  alternative.  Because  of  its  exceedingly  ex¬ 
perimental  and  evaluative  nature.  Familiar  Letter 
Combinations  is  timed  as  a  power  test. 

Although  the  highest  reliability  estinnate  deter¬ 
mined  from  pretestings  was  low,  and  correlations 
with  other  tests  of  ESUwere  small  and  even  negative, 
Familiar  Letter  Combinations  was  used  as  a  test  of 
ESTT  Tn  the  final  testing  because  of  its  novel  content 
and  its  apparent  fit  to  the  ESU  hypothesis. 


Letter  "U"  is  a  test  of  E's  sensitivity  to  the 
presence  ot  a  specified  letter  in  words  under  speeded 
conditions.  It  is  based  upon  Thurstone's  test  Letter 
"A "  (1938b),  which  split  its  variance  among  factors 
IHat  Thurstone  called  perceptual,  number,  and  word 
factor.  Bechtoldt  (1947)  found  Letter  "A"  to  be  loaded 
highly  on  a  factor  with  a  test  ot  crossing-out  specific 
letters  on  a  page  of  regularly  spaced  letters.  He 
named  the  factor;  "speed  of  recognition  of  predeter¬ 
mined  symbols  in  context  of  discrete  distractors.  " 
Although  Cattell  names  the  factor  on  which  this  test 
is  loaded  "speed  of  symbol  discrimination"  (Cattell , 
1963),  and  Guilford  and  Hoepfner  (1963)  suggest  the 
factor  is  ESU,  French,  et  al.  (1963)  conclude  that  our 
knowledge  concerning  this  factor  is  not  at  all  clear 
since  several  "sub-factors"  tend  to  pull  together  in 
different  ways,  depending  upon  the  tests  included  in 
the  factor-analytic  battery.  In  general,  a  test  like 
Letter  "U"  is  often  found  in  strong  relation  to  percep- 
tual- speed  tests.  To  clarify  this  ambiguity,  not  only 
were  four  tests  designed  to  measure  ESU  included  in 
the  battery  idong  wi&  Letter  "U",  but  also  two  strong 
perceptual-speed  (EFU)  tests. 


The  test  Symbol  Identities  wa*  designed  as  a 
measure  of  ET  sensitivity  to*  the  identity  or  non¬ 
identity  of  paired  sets  of  numbers,  letters,  and  words, 
under  speeded  conditions.  It  is  essentially  parallel 
to  tests  of  EFU,  in  which  identity  of  pairs  of  figures 
is  in  question.  Symbol  Identities  is  the  only  ESU  test 
employed  that  cuts  across  all  the  possible  stimuli 
considered  appropriate  in  the  symbolic  domain. 


Symbol  Identities  is  similar  to  many  of  the  tests 
designed  to  measure  clerical  speed  and  accuracy;  E 
decides  whether  or  not  the  two  members  of  pairs  of 
symbolsets  are  the  same  or  different.  This  test,  like 
Letter  "U",  could  conceivably  share  much  figural 
variance,  as  Es  could  compare  each  symbol  stimulus, 
figure  by  figure,  and  arrive  at  an  accurate  judgment. 
Such  activity  is  very  inefficient,  however;  a  figural 
attack  upon  Symbol  Identities  should  result  in  poor 
performance,  unless  it  I*  'used  only  when  a  quick 
symbolic  attack  does  not  yield  a  decisive  choice. 


ESC  Tests 

Four  experimental  tests  were  developed  to  mea¬ 
sure  the  factor  of  evaluation  of  symbolic  classes.  A 
symbolic  class  was  defined  for  this  Investigation  as  a 
group  of  symbols  with  some  common  property.  Such 
a  group  of  symbols  would  be  composed  of  at  least  two 
menpbers  whose  common  property  must  be  symbolic, 
not  figural  or  easily  semanticized.  Two  of  the  four 
tests  designed  to  measure  ESC  were  symbolic  adap¬ 
tations  of  tests  of  semantic  evaluation  of  classes;  an¬ 
other  test  was  rewritten  from  a  test  that  had  no  firm 
construct  "home";  and  the  remaining  test  was  con¬ 
structed  originally  for  this  investigation.  Tests  of 
symbolic  classes  previously  employed  such  stimuli 
as  number  groupings  (CSC  and  DSC)  and  number  pairs 
(CSC)  (Guilford  and  Hoepfner,  1963). 

Best  Number  Class,  the  new  test  developed  for 
this  study,  has  no  history  in  previous  studies  of  intel¬ 
ligence.  E's  task  is  to  assign  given  numbers  into  one 
of  four  classes  in  such  a  way  as  to  maximize  each 
number's  value  by  assigning  it  to  the  most  exclusive 
class  it  fits.  The  four  classes  into  which  the  stimuli 
were  to  be  assigned  were,  in  order  of  exclusiveness; 
EVEN  MULTIPLES.  ODD  MULTIPLES.  SQUARES, 
and  PRIMES.  Es  were  warned  that  the  numbers  could 
possibly  beassigned  to  several  classes  and  that  credit 
could  only  be  earned  by  assigning  each  number  to  its 
most  exclusive  class. 

The  test  Best  Number  Pairs  is  the  other  hypo¬ 
thesized  ESC  measure  employing  numbers  as  stimuli. 
The  item  format  is  adapted  from  the  test  Best  Word 
Pairs,  found  to  be  loaded,  in  part,  on  the  factor  EmG, 
evaluation  of  semantic  classes  (Nihira,  et  al.  ,  1964). 
In  the  symbolic  form,  E's  task  is  to  choose  one  of 
three  pairs  of  numbers  that  makes  the  best  class.  In 
order  from  best  to  poorest,  the  classes  are;  pairs  of 
perfect  squares,  pairs  of  multiples  of  the  same  num¬ 
ber,  pairs  of  odd  or  even  numbers,  and  pairs  with  no 
class  property.  Although  the  number  classes  in  this 
test  are  similar  to  the  classes  in  the  test  Best  Num¬ 
ber  Class,  the  task  in  this  test  is  close  to  a  sensitiv¬ 
ity-type  of  test,  whereas  the  task  called  for  in  Best 
Number  Class  is  close  to  the  estimation  type,  based 
upon  an  understanding  and  rnemory  of  the  number 
classes.  Like  its  semantic  counterpart.  Best  Num¬ 
ber  Pairs  was  developed  as  a  power  test. 

Sound  Grouping  is  a  test  with  a  long  history.  In 
each  item,  four  words  are  given,  three  of  which  are 
fairly  good  rhymes  and  the  fourth  is  not.  The  latter 
is  to  be  noted  and  selected,  for  the  right  answer.  The 
test's  factorial  composition  has  been  open  to  consid¬ 
erable  question  because  of  its  tendency  to  go  with. dif¬ 
ferent  factors,  depending  upon  the  battery  in  which  it 
has  been  analyzed.  Thurstone  found  that  its  variance 
split  over  his  factors  of  word  fluency,  spatial,  and 
verbal  relations  (1938a).  1-ater,  the  test  was  em¬ 
ployed  in  research  at  the  University  of  Southern  Cal¬ 
ifornia,  and  was  variously  found  to  split  its  variance 
over  the  factors  of  verbal  comprehension  (CMU)  and 
general  reasoning  (CMS)  (Green,  et  al.  ,  1953);  verbal 
comprehension  and  eduction  of  correlates  (NSR) 
(Guilford,  et  al. ,  1954);  and  eduction  of  correlates, 
facility  with  verbal  relations,  and  perceptual  speed 
(EFU)  (Hertzka,  et  al.  ,  1954). 


Because  of  this  history  of  factor  instability  and 
the  fact  that  the  above-mentioned  studies  did  not  in¬ 
clude  tests  of  what  would  now  be  called  symbolic 
classes,  Sound  Groyning  wa*  hypothesised  to  measure 
ESC,  a  seemingly  logical  place  for  it.  It  was  not  ex¬ 
pected,  however,  that  Sound  Grouping  would  sudden¬ 
ly  become  a  uni-factor  test  when  placed  in  a  battery 
with  several  ESC  tests. 
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There  Is  some  evidence  that  Sound  Grouping 
may  measure  some  flgural  ability,  as  Hertzka,  et  aH 
(1954)  suggest  in  their  explanation  of  its  loading  on 
the  perceptual- speed  factor.  It  would  be  in  an  audi¬ 
tory-units  category,  however,  rather  than  visual.  The 
four  word  stimuli  for  each  item  must  be  processed  as 
auditory  figural  units  before  the  word  with  the  differ¬ 
ent  sound  can  be  determined.  Once  the  figural  stimuli 
are  before  E,  the  task  is  simply  one  of  cognition  of 
the  stimulus  that  does  not  belong.  The  symbolic- 
evaluation  process  probably  enters  at  the  stage  of 
translating  the  visual  symbolic  and  semantic  inform¬ 
ation  into  auditory  figural  information,  based  upon 
the  varying  sounds  of  letters  in  combination.  It  was 
hypothesized  that  the  ability  to  make  the  translations 
rapidly  is  facilitated  by  symbolic  sensitivity  to  audi¬ 
tory  figural  correlates  of  letter  combinations.  This 
detailed  logical  analysis  resulted  in  the  hypothesis 
that  Sound  Grouping  might  share  considerable  ESC 
variance.  Several  pretestings  confirmed  the  idea  in 
that  Sound  Grouping  did  correlate  slightly  with  other 
ESC  tests. 

The  fourth  test  designed  to  measure  ESC  is 
Word  Choice.  The  item  format  was  adapted  from  the 
semantic  classes  test.  Glass  Name  Selection  (EMC),’ 
which  was  developed  by  the  first  author  and  which 
aided  in  the  definition  of  the  EMC  factor  in  the  study 
by  Nihira,  et  al.  (1964).  Whereas  in  the  semantic 
test  E  is  to  choose  the  best  class  name  for  a  group 
of  four  words,  in  the  symbolic  test  E  is  to  choose  the 
best  of  three  possible  additions  to  a  class  of  three 
words.  The  class  properties  used  in  Word  Choice  are 
symbolic,  e.  g.  ,  order  or  nearness  of  certain  letters 
or  types  of  letters  in  the  words.  This  test  also  dif¬ 
fered  from  its  semantic  counterpart  in  that  none  of 
the  alternative  words  for  any  class  completely  pos¬ 
sessed  all  the  class  properties;  a  best  word  had  to  be 
chosen,  even  though  it  was  slightly  wrong.  It  is  thus 
an  estimation  test. 

ESR  Tests 

The  four  tests  developed  to  measure  the  factor 
ESR,  evaluation  of  symbolic  relations,  employed  rec¬ 
ognized  connections,  based  upon  symbolic  variables, 
between  symbolic  units.  Examples  of  connections 
based  upon  symbolic  variables  are  "bigger  than," 
"equal  number  of  consonants,  "  and  "similar  ratios.  " 
Although  ordering  tasks  in  the  symbolic  area  appear 
often  ^o  be  more  relational  than  systemic,  consider¬ 
able  difficulty  in  developing  evaluative  forms  of  the 
task  precluded  development  of  such  tests  to  parallel 
more  closely  the  tests  of  other  symbolic-relations 
factors.  It  must  be  remembered,  that  as  with  the 
product  of  classes,  the  perceptual  stimuli  themselves 
need  not  be  symbolic,  but  the  connections  between  the 
stimuli  must  be  based  upon  some  symbolic  aspect  of 
the  stimuli. 

In  the  divergent-production  domain  of  thinking 
processes,  Gershon,  et  al.  (1963)  states  that  it  is 
easier  to  Invent  relational  tasks  that  require  the  in¬ 
troduction  of  alternative  relationships  amoi^  symbols 
than  it  is  to  invent  completion-production  items  as 
are  so  frequently  seen  in  the  semantic  tests  of  rela¬ 
tions.  Although  con^letlon-production  formats  are 
not  typically  relevant  to  tests  of  evaluation  abilities, 
the  task  restrictions  in  divergent-production  proces¬ 
ses  are  different  from  those  for  evaluative  processes. 
Two  of  die  experimental  ESR  tests  Introduce  alterna¬ 
tive  relationships,  while  the  other  two  Involve  evalu¬ 
ation  of  a  relation  in  terms  of  provided  rules. 


Stimuli  employed  in  previous  symbolic-relations 
testa  have  induded;  word  trends  and  alphabetical 
relationships  (CSR);  number  pairs  and  word  pairs 
(MSR);  letter  groups  and  numerical  equations  (DSR); 


word  pairs  and  letter  series  (NSR);  and  equations 
(ESR)  (Guilford  and  Hoepfner,  1963).  In  addition  to 
employing  numerical  equations  as  test  stimuli  for 
ESR  tests,  words  were  employed  in  two  teats,  so  that 
the  four  tests  were  equally  divided  between  number- 
letter  and  word  stimuli, 

The  first  experimental  ESR  test.  Related  Words 
I,  was  adapted  by  analogy  to  Matched  Verbal  Rela- 
Fions,  designed  for  factor  EMR,  evaluation  of  seman¬ 
tic  relations,  on  which  it  was  loaded  in  two  studies 
(Nihira,  et  al.  ,  1964;  Petersen,  et  al.  ,  1963).  In 
Related  Words  I,  E  must  estimate  which  of  three  al- 
ternatlve  word  pairs  is  most  similar  to  a  given  rela¬ 
ted  pair.  The  relation  between  members  of  any  pair 
is  based  upon  the  order  and  position  of  letters  and  the 
vowels  and  consonants  that  are  changed  or  moved. 
This  is  the  only  ESR  test  in  which  no  alternative  an¬ 
swer  is  completely  correct;  only  a  best  alternative  is 
to  be  selected.  Like  the  other  ESR  tests.  Related 
Words  I  was  designed  to  be  a  power  test. 

Sign  Changes  II  had  been  developed  as  an  ESR 
.test  to  be  used  in  a  predictive  battery  for  success 
in  ninth-grade  mathematics  courses  (Petersen,  et 
al.  ,  1963).  It  was  not  factor  analyzed,  but 

multiple-prediction  studies  did  indicate  that  the  test 
contributed  unique  variance  to  prediction  of  success 
to  a  significant  degree.  The  task  in  this  test  is  to 
determine  what  sign  changes,  if  any,  must  be  made  to 
change  a  numerical  expression  into  an  equation.  An 
elementary  understanding  of  arithmetical  operations 
and  the  relationships  of  equality  and  inequality  of  ex¬ 
pressions  is  all  E  needs  in  order  to  understand  clear¬ 
ly  the  test  items  and  the  task.  Because,  for  each 
item,  there  is  a  unique,  correct  answer,  and  because 
the  test  is  not  speeded,  for  Es  well-versed  in  such 
algebraic -equation  manipulating  the  evaluation  vari¬ 
ance  might  be  minimized  in  favor  of  variance  deter¬ 
mined  by  individual  differences  in  a  convergent-pro¬ 
duction  activity.  Results  of  the  Petersen,  et  al.  , 
(1963)  study,  however,  showed  that  the  correlations 
between  Sign  Changes  II  and  convergent-production 
tests  were  quite  lowT  It  was  hypothesized  that  even 
at  a  somewhatadvanced  stage  of  algebraic  competence 
(in  an  accelerated-algbera  group),  a  sensitivity  to  the 
correct  relationships  is  at  the  basis  of  the  process 
necessary  to  perform  on  this  test. 

Similar  Pairs  is  a  new  test,  in  both  idea  and 
items.  The  stimuli  are  word  pairs,  the  menibers  of 
which  are  related  by  letter  locations  and  letter  clianges. 
E's  task  is  to  judge  whether  the  members  in  two  such 
pairs  are  or  are  not  similarly  related.  The  process 
involved  in  responding  to  this  test  is  sensitivity  to 
sameness  or  differentness  of  the  relations  within  the 
word  pairs.  In  all  the  items,  the  relations  within  the 
pairs  were  kept  extremely  simple,  so  that  there  would 
be  little  or  no  difficulty  in  cognizing  the  relationships, 
so  that  cognition  variance  would  be  minimized  in  the 
test  scores  and  the reby  maximize  the  relative  impor¬ 
tance  of  the  evaluation  variance. 

Symbol  Manipulation  is  a  test  of  the  ability  to 
decide  wlietber  a  given  relationship  between  two  let¬ 
ters  follows  logically  from  other  statements  of  rela¬ 
tionship  involving  tte  same  letters,  where  the  rela¬ 
tionships  are  "greater  than,  "  "equal  to,"  and  "less 
than,  "and  their  negations,  all  statements  in  symbolic 
form.  This  test  has  appeared  in  five  previous  analy¬ 
ses  done  by  the  Aptitudes  Project.  In  two  analyses  it 
had  no  significant  loadings  on  any  factors,  probably 
because  there  was  a  paucity  of  symbolic  tests  and, 
more  particularly,  because  the  right  symbolic  factor 
was  not  determined  (Berger,  et  al. ,  1957;  Wilson,  et 
al. ,  1954).  In  tlie  first  analysis  in  vdiich  the  test  had 
significant  loadings  (Green,  et  al. ,  1953),  they  were 
on  a  factor  called  "logical  reasoning"  (later,  logical 
evaluation,  and  eventually  identified  as  EMR)  and  on 
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a  factor  called  "symbol  substitution,  "  which  might 
have  been  the  same  as  one  later  called  "symbolic 
manipulation"  for  the  test  of  the  same  name.  In  the 
second  analysis  (Guilford,  et  al.  ,  1954),  Symbol  Sub- 
stitution  again  shared  its  variance  with  the  logical- 
evaluation  and  symbol-manipulation  factors.  In  a 
third  analysis  (Kettner,  et  al.  ,  1959),  Symbol  Mani- 
pulation  again  led  uniquely  on  a  factor  given  the  same 
name.  In  the  same  analysis  a  symbol- substitution 
factor  was  found  separated  from  symbol  manipulation, 
wh  ich  lends  some  support  to  a  conclusion  stated  above 
about  an  earlier  symbol r substitution  factor.  It  was 
the  parallel  nature  of  Symbol  Man ipulation and  verbal- 
syllogistic  tests  of  logical  evaluation,  recognized  as 
EMR,  and  the  fact  that  this  test  also  showed  some 
variance  in  EMR  (probably  due  to  E's  verbalizing  the 
statements)  that  led  Guilford  and  Hoepfner  (1963)  tp 
recommend  the  test  for  ESR. 

ESS  Tests 

Like  the  tests  hypothesized  to  measure  ESU, 
tests  for  ESS,  evaluation  of  symbolic  systems,  seemed 
to  be  easy  to  construct.  Almost  one  dozen  tests  were 
developed  to  measure  ESS  and  were  pretested.  Most 
of  the  tests  at  this  experimental  stage  proved  to  Ivive 
reasonably  good  reliability  and  reasonable  intra-factor 
correlations.  The  five  tests  chosen  to  define  the  sys¬ 
tems  factor  in  the  final  analysis  broadly  cover  the 
various  types  of  symbolic  content  and  sensitivity  vs. 
estimation. 

All  the  stimuli  for  tests  of  symbolic  systems 
are  organized  aggregates  of  units  or  relations  wherein 
the  interrelated  or  interacting  parts  are  symbolically 
defined  within  the  aggregate.  The  system,  then,  is 
the  organization  or  pattern  of  parts,  which  may  be 
compared  with  another  system  as  to  identity  or  simi¬ 
larity  or  evaluated  for  internal  consistency.  Other 
tests  known  to  measure  factors  of  symbolic  systems 
have  employed:  sequences  and  alphabetical  patterns 
(CSS);  number-letter  codes  and  equations  (OSS);  and 
sequences  of  numerical  operations  and  of  letter  oper¬ 
ations  (NSS)  (Guilford  and  Hoepfner,  1963). 

Best  Letter  Set  was  designed  as  a  measure  of 
E's  ability  to  estimate  which  of  three  sets  of  three  or 
four  letters  each  is  most  like  a  given  set.  The  cri¬ 
terion  of  similarity  is  based  upon  the  order  and  kinds 
of  letters  within  the  set.  Although  such  small  sets  of 
letters  might  appear  to  function  as  units,  the  systems 
qualities  of  the  alternative  sets  were  sufficiently  sim¬ 
ilar  to  force  E  to  focus  on  them.  It  seemed  highly 
unlikely  that  even  the  most  sophisticated  E  could  treat 
the  stimuli  as  units  and  obtain  a  high  score  on  this 
test. 

The  next  two  tests  are  somewhat  alike  and  will 
be  discussed  together.  Both  Correct  Letter  Orders 
and  Correct  Number  Series  are  tests  of  E's  sensitiv- 
ity  to  internal  inconsistencies  in  symbolic  systems. 
The  stimuli  in  both  tests  are  sequences  of  symbols 
organized  according  to  some  simple  principle,  simi¬ 
lar  to  items  in  familiar  number-series  tests.  The 
systematic  order  is  stated  verbally  and  E  is  to  judge 
whether  or  not  the  sequence  follows  that  principle. 
This  test  uses  as  stimuli,  sequences  like  those  used 
by  Thurstone  (1938a,  1940).  Thurstone's  tests  were 
of  the  completion  variety  that  emphasize  the  cognition 
aspect  of  the  task;  E  cognizing  the  systematic  se¬ 
quence  and  showing  that  it  is  cognized  by  continuing 
the  sequence  in  a  consistent  manner.  Thurstone's 
tests,  in  both  letter  and  number  forms,  were  loaded 
on  factors  called  deductive  and  inductive.  More  re¬ 
cently,  series-type  tests  have  been  found  loaded  on 
the  factors  of  general  reasoning  (CMS),  eduction  of 
patterns  (CSS),  and  symbol  manipulation  (Kettner,  et 
a).,  1959);  and  on  a  factor  called  NSR  (Petersen,  et 
al.  ,  1963),  which  was  defined  in  part  by  tests  of  CSS 
and  N.S.S, 


The  test  Series  Relations  might  also  be  consid¬ 
ered  an  evaluative  form  of  a  number-series  test,  even 
though  the  task  appears  to  be  quite  unlike  that  for 
Correct  Number  Series.  In  Series  Relations,  E  is 
given  a  series  of  three  numbers  and  is  told  that  each 
element  of  the  series  except  the  first  one  is  deter¬ 
mined  from  the  previous  element  (one  to  the  left)  ac¬ 
cording  to  some  unknown  rule.  E  is  then  to  estimate 
which  of  three  alternative  rules  or  operations  would 
best  relate  each  series  element  to  the  previous  one. 
Although  E  might  simply  try  each  rule  upon  the  first 
and  second  series  elements,  obtain  a  three-number 
series,  and  compare  it  to  the  given  one,  selecting  the 
correct  rule,  he  is  forced  into  making  a  choice  or 
judgment  because  none  of  the  three  alternative  rules 
is  fully  correct.  That  is,  no  one  rule  will  correctly 
reconstruct  the  seiies  from  the  first  element;  but  one 
will  do  the  job  best. 

In  the  test  Way-Out  Numbers,  E  is  presented 
with  a  list  of  four  ordered  numbers  and  is  instructed 
to  choose  either  the  first  or  last  one  on  the  basis  of 
its  being  farther  away  from  the  remaining  three  num¬ 
bers.  In  other  words,  E  is  to  arrange  the  numbers 
on  the  dimension  of  numerical  value  and  is  to  choose 
that  extreme  number  whose  value  is  farther  from  the 
other  numbers'  values. 

E  is  provided  with  an  example  of  a  graphic  so¬ 
lution  in  the  instructions  and  is  also  told  that  there 
are  mathematical  methods  of  solution  (compute  the 
centroid  of  the  first  three  numbers  and  determine 
its  distance  from  the  last,  then  do  the  same  thing  for 
the  last  three  numbers,  comparing  the  two  distances). 
To  force  E  into  making  rapid  estimations,  he  is  told 
that  either  procedure,  while  yielding  correct  answers, 
would  be  inefficient  in  terms  of  the  time  spent  on  each 
item.  There  was  no  evidence  from  the  pretesting 
results  that  Es  ignored  this  advice. 

All  the  stimuli  used  in  this  test  formed  highly 
unevenly  distributed  series,  which  should  make  a 
number-estimation  task  possible.  After  the  test  had 
been  printed,  however,  it  was  discovered  that  there 
was  an  easier  method  than  roughly  estimating  in  order 
to  make  correct  responses.  Owing  to  lack  of  fore¬ 
sight  and  consequent  item  revision,  it  was  discovered 
that  the  correct  responses  could  be  chosen  simply  by 
subtracting  the  extreme  numbers  from  their  adjacent 
numbers.  The  larger  difference,  then,  indicated  the 
number  farther  away.  If  many  Es  discovered  this 
principle  and  employed  it,  in  spite  of  instructions,  it 
would  be  expected  that  Way-Out  Numbers  would  share 
its  variance  with  the  numerical-facility  factor  (MSI), 
dub  to  the  contribution  of  correct  subtraction. 

EST  Tests 

Three  experimental  tests  were  selected  to  mea¬ 
sure  the  factor  EST,  evaluation  of  symbolic  transfor¬ 
mations,  The  extreme  difficulty  of  constructing 
reliable  evaluative  tests  of  symbolic  transformations 
limited  the  choice  of  tests.  It  was  expected,  however, 
that  the  test  Derivations,  hypothesized  as  a  measure 
of  ESU,  might  aid  IndeHning  the  transformations  fac¬ 
tor.  The  content  of  the  three  tests  was  concerned 
with  changes  from  one  form  of  symbol  to  another 
equivalent  form,  or  changes  in  symbolic  units  to  meet 
certain  requirements.  The  transformations  tests 
developed  for  this  study  used  letters  and  words  as 
stimuli.  It  appeared,  during  test  construction,  that 
numerical  stimuli  were  not  readily  susceptible  to 
transformations  without  the  involvement  of  other  pro¬ 
ducts,  such  as  relations  or  systems.  The  denotative 
inflexibility  of  numbers  did  not  allow  for  equivalent 
forms  of  the  same  numerical  value  using  two  different 
symbols.  This  limitation  also  obtained  in  the  study 
isolating  the  only  other  known  symbolic-transformation 
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factor,  NST  (Guilford,  et  al. ,  1961).  The  tests  load¬ 
ing  on  the  NST  factor  all  involved  words  as  the 
stimuli. 

.  A  rather  common  transformation  of  words  and 
letters  is  any  code  that  allows  their  encoding.  In  al¬ 
most  all  cases  of  symbol  coding,  the  transformation 
of  one  set  of  symbols  to  its  encoded  set  of  symbols  is 
a  one-to-one  mapping  of  the  symbol  set  onto  the  code 
set.  Such  a  one-to-one  mapping  is  suitable  for  a  test 
of  sensitivity  to  errors  in  coding  only  when  the  test  is 
speeded  and  the  coding  system  is  well  known  by  the 
Es.  This  implies  that  the  sensitivity  to  slight,  but 
possibly  important  miscodings  is  an  evaluative  pro¬ 
cess  for  the  individual  udio  functions  well  (is  experi¬ 
enced)  in  the  coding  process. 

Because  no  coding  system  is  known  with  great 
generality  within  the  population,  and  because  it  is  in¬ 
efficient  and  self-defeating  to  teach  Es  a  complete 
coding  system  (memory  factors  might  predominate), 
the  EST  test.  Decoding,  employs  a  simple  and  am¬ 
biguous  code.  Simplicity  and  ambiguity  were  intro¬ 
duced  into  the  coding  system  for  Decoding  by  employ¬ 
ing  a  code  for  letters  which  does  not  map  one-to-one 
onto  the  alphabet.  The  ambiguity  of  the  code  allows 
for  words  to  be  judged  according  to  their  ease  of  en¬ 
coding  or  decoding.  The  change  from  an  unambiguous 
code  to  an  ambiguous  one  also  changes  the  type  of 
evaluation  test  involved.  Whereas  an  unambiguous 
code  and  experience  call  for  sensitivity,  anambiguous 
code  calls  for  estimation;  the  code  provides  incom¬ 
plete  information,  and  E  must  estimate  the  complete 
information. 

In  the  test  Decoding,  E  is  presented  with  two 
words  and  is  asked  to  choose  which  one,  if  coded, 
would  be  easier  to  decode  unambiguously.  E  is  also 
given  the  opportunity  to  judge  both  words  as  equal  in 
difficulty  of  decoding. 

Jumbled  Words  is  the  only  test  designed  for  EST 
that  is  in  the  sensitivity  category.  E  is  given  a  stim¬ 
ulus  word  containing  between  five  and  seven  letters 
and  is  to  judge  whether  or  not  each  of  five  adternative 
words  is  an  accurate  anagrammatic  derivation  from 
the  given  word.  Jumbled  Words  is,  therefore,  similar 
in  stimulus  material  to  the  test  Derivations,  which 
also  uses  anagram-typ'e  stimuli.  Pretesting  indicated 
that  Jumbled  Words  correlated  higher  with  EST  tests 
than  with  Derivations.  It  was  thought  that  this  corre¬ 
lational  pattern  was  due  to  the  fact  that  Jumbled  Words 
employed  all  the  letters  of  each  wor3^  so  that  the 
whole  word  had  to  be  evaluated  for  errors  in  trans¬ 
formation.  This  criterion  is  not  present  in  Deriva- 
tions,  wherein  only  the  same  letters  as  those  in  the 
given  word  were  the  criterion,  and  fewer  could  be 
used. 

The  third  test  designed  for  EST,  Typing  Errors, 
is  similar  to  Decoding  in  the  task  involved  and  the 
stimuli  used.  E  Is  given  an  incorrectly  typed  word 
and  is  to  choose  from  among  alternatives  the  word 
that  the  Incorrectly  typed  word  would  most  likely  be. 
The  judgments  are  made  on  the  basis  of  common  typ¬ 
ing  errors,  due  to  the  arrangement  of  the  typewriter 
keyboard.  A  keyboard  diagram  is  printed  on  each 
test  page  for  E’s  reference. 

The  estimation  process  involved  in  responding 
to  Typing  Errors  is  probably  not  dependent  upon  EST 
ability  alone,  however.  It  wpuld  seem  that  some  flg- 
ural  ability  would  be  involved  in  this  test  due  to  the 
spatial  nature  of  the  keyboard  arrangement.  Further, 
it  might  be  expected  that  typing  experience  might 
enter  into  proficiency  at  the  required  Usk.  To  deter¬ 
mine  to  some  degree  the  relative  contributions  of 
these  two  extraneous  influences,  it  was  decided  to  in¬ 
clude  at  least  one  flgural  factor  in  the  factor  analysis 
and  also  to  correlate  test  scores  with  a  measure  of 
typing  experience. 


ESI  Tests 

Tests  designed  to  measure  the  factor  ESI,  eval¬ 
uation  of  symbolic  implications,  employed  all  types 
of  symbolic  stimuli  in  the  list.  For  the  evaluation 
process,  implications  are  defined  as  the  expectancies 
or  probable  relative  values  of  the  presented  symbols 
(estimation),  or  possible  symbolic  interpretations  of 
a  unit  or  system  (sensitivity  to  symbolic  problems). 
Previous  investigations  of  symbolic-implications  fac¬ 
tors  have  employed  the  following  stimuli  in  the  tests: 
crossword  puzzles  and  disordered  symbols  (CSI); 
simple  numerical  problems  (MSI);  words  and  equa¬ 
tions  (DSI);  and  simple  equations  (NSI)  (Guilford  and 
Hoepfner,  1963).  The  ESI  tests  developed  for  this 
study  were  based  upon  the  stimuli  used  successfully 
previously  and  upon  novel  stimuli  that  appeared  ap¬ 
propriate  for  evaluative  processing. 

The  test  Abbreviations  presented  E  with  a  short¬ 
ened  spelling  of  a  common  word,  E  to  choose  one  of 
the  three  alternative  words  that  the  abbreviated  word 
most  likely  implies.  The  meanings  of  the  words  are 
irrelevant  to  choosing  an  alternative,  and  the  spelling 
of  the  alternatives  is  correct.  The  only  task  for  E  is 
to  choose  the  most  expected  value  for  the  abbrevia¬ 
tion,  a  task  of  estimating.  No  observance  of  short¬ 
hand  principles  was  exercised  in  test  construction; 
the  abbreviations  were  short  and  relatively  unambig¬ 
uous.  Usually,  but  not  always,  this  implied  dropping 
vowels  and  unsounded  consonants  from  the  keyed  al¬ 
ternative.  E  was  warned,  however,  that  sounding-out 
the  abbreviation  would  not  necessarily  aid  him  in  his 
choice . 

Abbreviations  had  been  employed  in  a  previous 
study  at  the  University  of  Southern  California  (Peter¬ 
sen,  et  al.  ,  1963),  but  was  not  factor  analyzed.  Its 
highest  correlations  in  the  two  analyses  reported 
were  with  the  test  Correlate  Completion  II,  a  test  of 
factor  NSR.  Its  reliability  was  reported  as  .  47. 
Abbreviations  was  item-analyzed  and  lengthened  for 
this  investigation  in  order  to  improve  its  reliability. 

Letter  Problems  is  similar  in  format  to  Form 
Reasoning,  a  test  ot  bISI.  In  Form  Reasoning,  E 
makes  appropriate  substitutions  of  symbols  and  solves 
a  simple  equation.  The  evaluation  form  uses  letters 
as  the  stimuli  and  asks  E  not  to  solve  the  equation, 
but  to  judge  the  difficulty  or  possibility  of  solving  it, 
on  the  basis  of  provided  rules.  It  was  hypothesized 
that  E  would  lave  to  make  his  judgments  based  on 
foresight.  E's  judgments  were  of  the  three-category 
type;  problems  were  easy  to  solve  (straightforward), 
difficult  to  solve  (involving  manipulations),  or  impos¬ 
sible  to  solve  due  to  inadequacies  of  the  table  of  sub-, 
stltutions. 

The  third  ESI  test  is  named  S  Test.  The  "S"  in 
the  name  means  that  the  test  is  symbolic.  In  this  test 
E  is  given  a  stimulus  about  which  he  is  to  find  a  prob¬ 
lem  to  solve.  The  solution  indicates  the  nature  of  the 
problem  to  which  E  was  sensitive. 

TheS  Test  is  anadaptation  of  the  F  Test  (Wilson, 
et  al.,  1954),  designed  as  a  test  of  senslllvlty  to  prob¬ 
lems,  which  had  its  highest  (but  insignificant)  loading 
on  the  originality  (DMT)  factor  Instead.  Although  its 
behavior  was  not  according  to  hypothesis,  it  was  de¬ 
cided  to  revise  the  test  to  lead  E  into  symbolic  re¬ 
sponses  in  all  items;  some  were  previously  verbal. 
Then  the  test  might  measure  E's  sensitivity  to  sym¬ 
bolic  implications  of  unstructured  problems.  It  should 
be  noted,  however,  that  this  test  is  not  congruent  with 
the  conception  that  evaluation  is  "sensitivity  to  error," 
for  no  error  is  judged.  It  is  a  test  of  E's  sensitivity 
to  implications  (as.  Indeed,  it  turned  out)  rather  than 
a  sensitivity  to  errors  in  implications. 
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factor,  NST  (Guilford,  et  al. ,  1961).  The  teata  load¬ 
ing  on  the  NST  factor  all  involved  words  as  the 
stimuli. 

.  A  rather  common  transformation  of  words  and 
letters  is  any  code  that  allows  their  encoding.  In  al¬ 
most  all  cases  of  symbol  coding,  the  transformation 
of  one  set  of  symbols  to  its  encoded  set  of  symbols  is 
a  one-to-one  mapping  of  the  symbol  set  onto  the  code 
set.  Such  a  one-to-one  mapping  is  suitable  for  a  test 
of  sensitivity  to  errors  in  coding  only  when  the  test  is 
speeded  and  the  coding  system  is  well  known  by  the 
Es.  This  implies  that  the  sensitivity  to  slight,  but 
possibly  important  mlscodings  is  an  evaluative  pro¬ 
cess  for  the  individual  who  functions  well  (is  experi¬ 
enced)  in  the  coding  process. 

Because  no  coding  system  is  known  with  great 
generality  within  the  population,  and  because  it  is  in¬ 
efficient  and  self-defeating  to  teach  Es  a  complete 
coding  system  (memory  factors  might  predominate), 
the  EST  test.  Decoding,  employs  a  simple  and  am¬ 
biguous  code.  Simplicity  and  ambiguity  were  intro¬ 
duced  into  the  coding  system  for  Decoding  by  employ¬ 
ing  a  code  for  letters  which  does  not  map  one-to-one 
onto  the  alphabet.  The  ambiguity  of  the  code  allows 
for  words  to  be  judged  according  to  their  ease  of  en¬ 
coding  or  decoding.  The  change  from  an  unambiguous 
code  to  an  ambiguous  one  also  changes  the  type  of 
evaluation  test  involved.  Whereas  an  unambiguous 
code  and  experience  call  for  sensitivity,  anambiguous 
code  calls  for  estimation:  the  code  provides  incom¬ 
plete  information,  and  E  must  estimate  the  complete 
information. 

In  the  test  Decoding,  E  is  presented  with  two 
words  and  is  asked  to  choose  which  one,  if  coded, 
would  be  easier  to  decode  unambiguously.  E  is  also 
given  the  opportunity  to  judge  both  words  as  equal  in 
difficulty  of  decoding. 

Jumbled  Words  is  the  only  test  designed  for  EST 
that  is  in  the  sensitivity  category.  E  is  given  a  stim¬ 
ulus  word  containing  between  five  and  seven  letters 
and  is  to  judge  whether  or  not  each  of  five  alternative 
words  is  an  accurate  anagrammatlc  derivation  from 
the  given  word.  Jumbled  Words  is,  therefore,  similar 
in  stimulus  material  to  the  test  Derivations,  which 
also  uses  anagram-typb  stimuli.  Pretesting  Indicated 
that  Jumbled  Words  correlated  higher  with  EST  tests 
than  with  Derivations.  It  was  thought  that  this  corre¬ 
lational  pattern  was  due  to  the  fact  that  Jumbled  Words 
employed  all  the  letters  of  each  wor3^  so  that  the 
whole  word  had  to  be  evaluated  for  errors  In  trans¬ 
formation.  This  criterion  is  not  present  in  Deriva¬ 
tions  ,  wherein  only  the  same  letters  as  those  in  the 
given  word  were  the  criterion,  and  fewer  could  be 
used. 

The  third  test  designed  for  EST,  Typing  Errors, 
is  similar  to  Decoding  in  the  task  invoiveo  and  the 
stimuli  used.  &  Is  given  an  incorrectly  typed  word 
and  is  to  choose  from  among  alternatives  the  word 
that  the  Incorrectly  typed  word  would  most  likely  be. 
The  judgments  are  made  on  the  basis  of  common  typ¬ 
ing  errora  due  to  the  arrangement  of  the  typewriter 
keyboard.  A  keyboard  diagram  is  printed  on  each 
test  page  for  E's  reference. 

The  estimation  process  involved  in  responding 
to  Typing  Errors  is  probably  not  dependent  upon  EST 
ability  alone,  however.  It  would  seem  that  some  fig- 
ural  ability  would  be  Involved  in  this  test  due  to  the 
spatial  nature  of  the  keyboard  arrangement.  Further, 
it  might  be  expected  that  typing  experience  might 
enter  into  proficiency  at  the  required  task.  To  deter¬ 
mine  to  some  degree  the  relative  contributions  of 
these  two  extraneous  influences.  It  was  decided  to  In¬ 
clude  at  least  one  flgural  factor  in  the  factor  analysis 
and  also  to  correlate  test  scores  with  a  measure  of 
typing  experience. 


ESI  Tests 

Tests  designed  to  measure  the  factor  ESI,  eval¬ 
uation  of  symbolic  implications,  employed  all  types 
of  symbolic  stimuli  in  the  list.  For  the  evaluation 
process,  implications  are  defined  as  the  expectancies 
or  probable  relative  values  of  the  presented  symbols 
(estimation),  or  possible  symbolic  interpretations  of 
a  unit  or  system  (sensitivity  to  symbolic  problems). 
Previous  investigations  of  symbolic- Implications  fac¬ 
tors  have  employed  the  following  stimuli  in  the  tests: 
crossword  puzzles  and  disordered  symbols  (CSI); 
simple  numerical  problems  (MSI);  words  and  equa¬ 
tions  (DSI);  and  simple  equations  (NSI)  (Guilford  and 
Hoepfner,  1963).  The  ESI  tests  developed  for  this 
study  were  based  upon  the  stimuli  used  successfully 
previously  and  upon  novel  stimuli  that  appeared  ap¬ 
propriate  for  evaluative  processing. 

The  test  Abbreviations  presented  E  with  a  short¬ 
ened  spelling  of  a  conunon  word,  E  to  choose  one  of 
the  three  alternative  words  that  the  abbreviated  word 
most  likely  implies.  The  meanings  of  the  words  are 
irrelevant  to  choosing  an  alternative,  and  the  spelling 
of  the  alternatives  is  correct.  The  only  task  for  E  is 
to  choose  the  most  expected  value  for  the  abbrevia¬ 
tion.  a  task  of  estimatlag.  <,  No  ^eervatMiS  of  .flwtt# 
hand  principles  was  exercised  In  test  cmistructlbn; 
the  abbreviations  were  short  and  relatively  unambig¬ 
uous.  Usually,  but  not  always,  this  implied  dropping 
vowels  and  unsounded  consonants  from  the  keyed  al¬ 
ternative.  E  was  warned,  however,  that  sounding-out 
the  abbreviation  would  not  necessarily  aid  him  in  his 
choice. 

Abbreviations  had  been  employed  in  a  previous 
study  at  the  bnlverslty  of  Southern  California  (Peter¬ 
sen,  et  al.  ,  1963),  but  was  not  factor  analysed.  Its 
highest  correlations  in  the  two  analyses  reported 
were  with  the  test  Correlate  Completion  11,  a  test  of 
factor  NSR.  Its  reliability  was  reported  as  .  47. 
Abbreviations  was  Item-analyzed  and  lengthened  for 
this  invesllgatlon  in  order  to  Improve  Its  reliability. 

Letter  Problems  Is  similar  in  format  to  Form 
Reasoning,  a  lest  of  NSI.  In  Form  Reasoning,  t 
makes  appropriate  substitutions  of  symbols  and  SMves 
a  simple  equation.  The  evalxiatlon  form  uses  letters 
as  the  stimuli  and  asks  E  not  to  solve  the  equation, 
but  to  judge  the  difficulty  or  possibility  of  solving  it, 
on  the  basis  of  provided  rules.  It  was  hypothesised 
that  E  would  have  to  make  his  judgments  based  on 
foresight.  E's  judgments  were  of  the  three- category 
type;  problems  were  easy  to  solve  (straightforward), 
difficult  to  solve  (Involving  manipulations),  or  Impos¬ 
sible  to  solve  due  to  inadequacies  of  the  table  of  sub¬ 
stitutions. 

The  third  ESI  test  Is  named  S  Test.  The  "S"  In 
the  name  means  that  the  test  Is  symbolic.  In  this  test 
E  is  given  a  stimulus  about  which  he  Is  to  find  a  prob¬ 
lem  to  solve.  Tbs  solution  Indicates  the  nature  ci  tte 
problem  to  which  E  was  sensitive. 

The  S  Test  Is  anadaptatlon  of  the  F  Test  (Wilson, 
et  al.,  1954).  designed  as  a  test  of  sensUlvliy  to  prob¬ 
lems,  which  had  its  highest  (but  Insignificant)  Ic^lng 
on  the  originality  (DMT)  factor  Instead.  Although  Its 
behavior  was  not  according  to  hypothesis.  It  was  de¬ 
cided  to  revise  the  test  to  lead  E  Into  syndMllc  re¬ 
sponses  In  all  items;  some  were  previously  verbal. 
Then  the  test  might  measure  E's  sensitivity  to  sym¬ 
bolic  Implications  of  unstructured  proUsms.  It  should 
be  noted,  however,  that  this  test  is  not  coi^rueat  with 
the  conception  that  evaluation  Is  "sensitivity  to  error," 
for  no  error  Is  Judged.  It  is  a  test  of  E's  sanstUvlty 
to  Implications  (as.  Indeed,  It  turned  out)  rather  tihan 
a  sensitivity  to  errors  In  Implications. 
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PROCEDURES 


The  Sample 

The  sample  utilized  in  this  study  consisted  of 
the  entire  senior -class  student  population  o{  the 
Claremont  High  School  of  Claremont,  California.  5 
Although  131  boys  and  180  girls  participated  in  the 
testing,  the  sample  was  later  reduced  to  86  boys  and 
139  girls,  for  whom  complete  test  data  for  all  exper¬ 
imental  factor  tests  were  available.  The  only  criteri¬ 
on  for  exclusion  of  Es  from  the  sample  was  incomplete 
data  on  these  measures. 

Age  and  IQ  information  was  available  for  219 
and  199  students,  respectively.  Generalizing  from 
such  demographic  data  available  for  most  of  the  sam¬ 
ple  utilized,  the  estimated  mean  age  was  17.4  years. 
The  estimated  mean  IQ,  computed  from  combinations 
of  scores  obtained  from  the  several  IQ  measures, 
which  were  variously  administered  between  the  eighth 
and  eleventh  grades,  was  110.4,  Although  IQ's  ranged 
from  80  to  151,  no  students  were  deleted  from  the 
sample  on  the  basis  of  extreme  indices  of  general  in¬ 
telligence.  The  test  response*  of  14  student*  with 
IQ's  below  95,  who  might  not  have  understood  the  test 
instructions,  were  carefully  examined  to  determine 
whether  or  not  lack  of  understanding  should  disqualify 
them  from  the  sample.  None  of  the  students,  however, 
demonstrated  consistently  poor  performance  on  all 
tests. 


Administration  of  the  Test  Battery 

The  total  sample  of  Es  was  tested  in  the  morn¬ 
ings  and  afternoons  of  Tuesday,  November  26,  and 
Wednesday,  November  27,  1963.  Each  testing  ses¬ 
sion  required  approximately  two  hours.  The  tests 
were  arranged  into  16>page  booklets  of  about  six  tests 
each.  The  tests  of  each  factor  were  so  arranged  that 
order  effects  and  fatigue  effects  would  be  approxi¬ 
mately  equal  for  all  factors  expected  to  be  demon¬ 
strated.  Each  booklet  required  approximately  50 
minutes  for  administration,  so  that  within  each  two- 
hour  testing  session,  there  was  sufficient  time  for 
distributing,  administering,  and  collecting  two  book¬ 
lets.  Short  recesses  were  given  between  booklets. 

In  both  of  two  separate  test  administrations,  the 
tests  were  administered  at  the  same  times  and  in  the 
same  order.  The  two  examiners  used  administration 
manuals  which  contained  introductory  and  orientation 
statements,  the  printed  test  instructions,  and  sug¬ 
gested  answers  to  questions  expected  to  be  raised  by 
the  Es.  The  test  instructions  were  read  aloud  by  the 
administrators  while  the  Es  maintained  pace  by  read¬ 
ing  the  instructions  silently. 


The  testing  conditions  under  which  the  battery 
was  administered  were  almost  ideal,  with  one  major 
exception.  The  days  of  the  test  administration, 
November  26  and  27,  unfortunately,  were  only  four 
and  five  days  after  the  tragic  death  of  President 
John  Fitzgerald  Kennedy.  Both  administrator*  and 
school  personnel  were  aware  that  after  the  day  of 
national  mourning,  the  preceding  Monday,  the  stu¬ 
dents  were  still  disturbed  and  restless.  The  effects 
of  the  national  tragedy  upon  the  results  of  this  study 
are  unknown. 


^For  the  splendid  cooperation  in  testing  in  the 
Claremont  High  School,  we  are  Indebted  to  Mr.  R.  H. 
Oyler,  Mr.  W.  D.  Wiley,  and  Miss  Catherine Flannelly 


Scoring 

Scoring  criteria  for  the  marker  teats  were  de¬ 
veloped  from  the  scoring  guides  employed  in  previous 
studies  with  onq  exception.  Scortog  criteria  for  the 
newly  developed  e]q>erlmental  tests  were  based  upon 
preliminary  results  of  pretestings  with  university 
students  in  undergraduate  psychology  courses.  The 
S  Test  was  scored  subjectively  by  a  trained  scorer  to 
determine  whether  or  not  each  response  indicated  E's 
sensitivity  to  a  symbolic  implication.  Every  test  was 
rescored  by  a  second  scorer. 


The  Appendix  describes  in  more  detail  the  scor¬ 
ing  procedures  for  each  test.  It  will  be  noted  that 
most  of  the  tests,  in  which  alternative  responses  are 
made  or  accepted,  have  a  scoring  formula  applied  to 
the  number  of  right  and  wrong  responses.  Hence¬ 
forth,  such  formula  scores  will  be  referred  to  as  raw 
scores,  as  all  analyses  were  performed  upon  those 
scores. 


STATISTICAL  ANALYSIS  AND  SOME  RESULTS 


Statistical  Treatment  of  the  Score  Data 

After  scoring,  the  part  and  total  scores  for  each 
test  were  punched  onto  IBM  cards.  The  two  rating 
variables,  one  containing  47  separate  ratings,  the 
other  containing  8  separate  ratings,  were  also  punched 
onto  IBM  cards.  All  school  measures  available  on  the 
Es  were  punched  onto  another  deck  of  cards. 


Frequency  distributions  were  obtained  for  all 
part  and  total  scores  to  determine  whether  or  not  the 
variables  would  meet  the  requirements  of  the 
Pearson-r  coefficient.  A  normalizing  transformation 
was  applied  to  those  variables  that  were  moderately 
skewed  or  exceedingly  platykurtic.  Extremely  skewed 
or  truncated  variables  were  dichotomized  near  their 
medians.  Descriptions  of  the  frequency  distribution* 
and  transformations  for  all  the  variables  are  listed  in 
Table  2. 


If  the  four  transformed  motivation  measures 
actutdly  do  measure  motivational  differences  over  a 
large  range,  ,  one  might  expect  the  optimum  principle 
of  motivation  to  be  working  (Vlnacke,  1960).  The  op¬ 
timum  principle  states  that  performance  I*  best  when 
motivation  is  at  an  intermediate  or  optimal  level.  A* 
motivation  decreases,  performance  decrease*  due  to 
lack  of  Interest  and  attention,  and  as  motivation  in¬ 
creases,  performance  decreases  due  to  the  Interfer¬ 
ence  of  anxiety.  This  principle  suggests  that  the 
relationships  between  the  motivation  measure*  and 
the  remaining  measures  of  intellectual  performance 
may  be  curvilinear.  To  check  this  possibility  before 
Intercorrelatlon*  were  computed,  scatter  plots  of 
several  intelligence-variable  scores  with  the  motiva¬ 
tion-variable  scores  were  Inspected.  No  curvUlnear- 
Ity  was  apparent  from  visual  inspection,  so  it  was 
assumed  that  the  optimum  principle  was  not  at  work 
to  affect  the  data,  probably  because  no  E's  motivation 
was  so  high  as  to  cause  emotional  Interference. 


19 


Table  2 

Distributions,  Means,  Standard  Deviations,  and  Reliabilities  of  Scores 


Teat 

Form  of 
Distribution^ 

Mean 

Standard 

Deviation 

Reliability*’ 

1. 

Abbreviations 

0  . 

11.  67 

4.  20 

.26* 

2. 

Best  Letter  Set 

13.  47 

5.  41 

.  56 

3. 

Best  Number  Class 

.. 

22.  67^' 

6.  74 

.  87 

4. 

Best  Number  Pairs 

0 

17.  32 

5.99 

.  73 

5. 

Camouflaged  Words 

0 

8.  33 

2.  89 

.  74® 

6. 

Circle  Reasoning 

- 

6.  84 

2.  72 

.  67e 

7. 

Correct  Letter  Orders 

+ 

16.  04 

8.  17 

.  58 

8. 

Correct  Number  Series 

0 

19.  66 

9.  67 

.  74 

9. 

Correct  Spelling 

0 

36.  98 

11. 49 

.  75 

Id. 

Decoding 

0 

16.  20 

6.  39 

.  74 

11. 

Derivations 

0 

99.  96 

20.  08 

.  81 

12. 

Disemvowelled  Words 

d 

11.  53 

4.  21 

.  79® 

13. 

Familiar  Letter  Combinations 

0 

14.  23 

6.  59 

.  43® 

14. 

Form  Reasoning 

-- 

17.  49^’ 

5.  26 

.  96® 

15. 

Identical  Forms 

0 

38.  12 

7.  30 

.  63l 

16. 

Jumbled  Words 

-- 

36.  83^5 

10.  86 

.  75 

17. 

Letter  Problems 

. 

14.  95= 

8.  88 

.  88 

18. 

Letter  Triangle 

+ 

5.  68 

2.  85 

.  55® 

19. 

Letter  "U" 

0 

53.  63 

11.  28 

.  84 

20. 

Marking  Speed 

+ 

94. 249 

18.  12 

.  44^ 

21. 

Number  Classification 

11.  41  = 

3.  35 

.  72® 

22. 

Number  Grouping 

- 

14.  10 

4.  42 

.  79 

23. 

Number-Group  Naming 

-- 

10.  17^ 

2.  12 

.  76® 

24, 

Numerical  Operations 

+ 

22.  23 

8.  62 

.  78 

25. 

Operations  Sequence 

0 

12.  27 

5.  29 

.  80 

26. 

Perceptual  Speed 

0 

48.  01 

9.  21 

.  65’ 

27. 

Related  Words  I 

14.  00 

5.  41 

.  55 

28. 

S  Test 

+ 

8.  55 

3.  42 

.69® 

29. 

Seeing  Trends  II 

0 

8.  16 

3.  25 

.  80® 

30. 

Series  Relations 

0 

9.  93 

7.  22 

.  74 

31. 

Sign  Changes 

- 

17.  49 

5.  05 

.57 

32. 

Sign  Changes  II 

17.  31’’ 

3.  64 

.  82 

33. 

Similar  Pairs 

20.  14= 

7.  40 

.  74 

34. 

Sound  Grouping 

+ 

11.  92 

6.  59 

.  74 

35. 

Symbol  Grouping 

0 

11.  22 

4.  84 

.  848 

36. 

Symbol  Identities 

0 

72.  12 

14.21 

.90 

37. 

Symbol  Manipulation 

-- 

21.40= 

8.  27 

.  74 

38. 

Symbol  Reasoning 

+ 

17.  95= 

10.29 

.  78 

39. 

Typing  Ertors 

+ 

9.  44= 

4.  99 

.  49 

40. 

Varied  Symbols 

0 

10.  92 

4.  50 

.67 

41. 

Way-Out  Numbers 

23.  48b 

6.  55 

.  76 

42. 

Word  Changes 

-- 

11.07= 

4.  47 

.  878 

43. 

Word  Choice 

0 

13.  05 

5.  81 

.62 

44. 

Word  Combinations 

+ 

10.  05= 

6.42 

.72 

45. 

Word  Patterns 

0 

70.  39 

9.  14 

.  75 

46. 

Word  Relations 

10.  76 

4.  64 

.  78 

47. 

Word  Transformation 

w 

26.  69 

7.91 

.838 

48. 

Rating  -  Test  Liking 

0 

100.01  = 

20.  71 

.90 

49. 

Rating  -  Booklet  Effort 

• 

200.06= 

65.28 

.95 

50. 

Sex 

+ 

.  38 

.49 

-- 

51. 

Prediction-Achievement  Discrepancy  0 

199.42 

59.  13 

.43’ 

52. 

ITEO  General  Vocabulary 

- 

20.  35 

5.45 

.81’ 

53. 

PSAT  Verbal 

0 

47.  88 

11.62 

.95’ 

54. 

SCAT  Verbal 

0 

304. 82 

14.  33 

.  85’ 

55. 

Mathematics  Experience 

+ 

3.  63 

2.22 

-- 

56. 

Typing  Experience 

+ 

1. 26 

.  81 

57. 

Shorthand  Experience 

++ 

.  24 

.  54 

-- 

58. 

Test  Administrator 

+ 

.  33 

.47 

-- 

‘  The  skewneas  of  score  dlatributions  la  deacrl^^  as  follow*:  +-f,  strong  positive  skew- 
neea;  *,  alight  poaltive  akewneaa;  0,  normal  diatribution;  alight  negative  akewneaa;  and 
atrong  negative  akewneaa. 


b  Total  acorea  dichotomized  at  the  mediana  for  Inter  cor  relatlona. 
c  Total  acorea  C>acaled  for  Intercorrelatlona. 

^All  eatlmatea  of  reliability  are  Spearman-Brown,  correctlona  of  correlation  between 
parta  unleaa  noted. 

‘Kuder-Richardaon  eatlmate  of  reliability. 

^  Communallty  entered  aa  reliability  eatlmate. 

SReliablllty  eatlmated  through  formula  21.  21,  in  Gulllkaen(1950). 
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After  It  was  ascertained  that  all  the  part-score 
data  and  total-score  data,  raw  or  scaled,  met  the  re¬ 
quirements  of  the  Pearson  r  or  its  approximations, 
reliability  estinaates  were  obtained  from  the  raw 
scores  of  the  tests.  For  all  tests  with  two  or  more 
parts,  Spearman-Brown  reliability  estimates  were 
computed.  Kuder-Richardson  estimates  of  reliability 
were  computed  for  all  one-part  tests  that  showed  no 
evidence  of  speeding.  Reliability  estimates  for  one- 
part  tests,  wherein  each  item  had  a  large  possible 
range  of  scores,  were  computed  by  a  formula  sug¬ 
gested  by  Gulliksen  (1950,  p.  378).  Reliabilities  of 
one-part  speeded  tests  and  of  school  measures  could 
not  be  estimated.  Their  communalities  were  expected 
to  approximate  the  necessary  estinaates  of  reliability. 


In  all  cases,  the  new  estimates  of  test  reliabili¬ 
ties  were  very  similar  to  those  obtained  from  previous 
studies  or  from  pretesting  results  where  tests  had 
been  used  before.  This  is  especially  interesting  in 
the  cases  of  the  marker  tests,  which  had  been  gener¬ 
ally  shortened.  The  reliabilities  reported  in  Table  2 
suggest  that  shortening  a  teston  empirical  bases  does 
not,  upon  readministration,  severely  reduce  its  relia¬ 
bility. 


Item  analyses  of  two  experimental  tests  that 
retained  low  reliability  estimates  from  pretesting 
(Abbreviations  and  Familiar  Letter  Combinations)  did 
not  improve  the  estimates,  so  the  tests  were  included 
in  the  analysis  without  item  deletions.  All  of  the  re¬ 
liability  estimates  reported  in  Table  2,  therefore,  are 
based  upon  the  tests  exactly  as  they  were  printed  and 
administered,  with  no  after-the-fact  item  deletions 
or  item  weightings.  In  general,  the  reliability  esti¬ 
mates  for  all  tests  were  considered  adequate  for  the 
factor-analysis  procedures  to  follow. 


The  means  and  standard  deviations  of  the  vari¬ 
ables  are  also  listed  in  Table  2.  In  all  but  three 
cases,  these  descriptive  statistics  are  based  upon  raw 
scores,  before  any  transformations  were  applied. 
Variables  48,  49,  and  51  were  automatically  standard¬ 
ized  in  their  computations  so  that  their  means  and 
standard  deviations  have  been  arbitrarily  set.  Few 
comparisons  between  the  descriptive  statistics  com¬ 
puted  from  this  analysis  and  those  from  other  analyses 
can  be  made,  either  because  the  tests  were  shortened 
and  otherwise  altered,  or  else  because  they  had  never 
before  been  administered  to  a  large  sample. 


Intercor  relations 

The  score  matrix  of  58  raw,  scaled,  and  dicho¬ 
tomized  variables  was  punched  onto  IBM  cards.  Be¬ 
cause  the  score  matrix  was  Incomplete  and  the  data 
were  differentially  scaled,  resulting  in  different  kinds 
of  correlation  coefficients,  the  correlation  matrix 
was  obtained  from  the  WDCORR  program  at  the 
Western  Data  Processing  Center.  This  program 
computes  correlation  coefficients  between  variables 
based  upon  the  total  number  of  individuals  for  whom 
scores  are  available.  Most  of  the  correUtlon  coef¬ 
ficients  In  Table  3,  therefore,  are  based  upon  the 
whole  sample  of  225  Es,  but  some  are  based  upon  the 


four  variables  for  which  not  all  Es'  scores  were 
available.  These  variables  and  the  number  of  scores 
available  are;  variable  51,  165  scores;  variable  52, 
187  scores;  variable  53,  109  scores;  and  variable  54, 
107  scores.  The  attenuated  samples  for  these  vari¬ 
ables  taken  independently,  of  course,  result  in  further 
attenuation  of  sample  size  for  the  coefficients  anaong 
them.  The  coefficient  between  variables  52  and  54 
was  computed  from  a  common  sample  of  only  66  Es. 
This  sample  size  was  the  smallest  from  which  any 
coefficients  were  computed  and  was  considerably 
smaller  than  the  next  smallest  ■  sample  of  87  for  the 
correlation  between  variables  52  and  53. 


Such  variations  in  sample  size  upon  which  cor¬ 
relation  coefficients  are  based  introduces  additional 
possibilities  of  error  into  the  correlation  matrix  due 
to  the  necessary  generalization  that  each  coefficient 
in  the  matr ix equally  estimates  the  actual  intercorre¬ 
lation  between  the  variables.  It  was  decided,  how¬ 
ever,  that  a  biased  estimate  would  be  better  than  none 
at  all,  and  since  the  Es  in  the  reduced  samples  ap¬ 
peared  to  have  been  selected  on  irrelevant  variables, 
i.  e.  ,  their  attendance  at  the  school  when  the  tests 
were  administered,  the  bias  was  thought  to  be  small. 


An  additional  consideration  of  the  WDCORR 
program  is  that  it  computes  product-moment  corre¬ 
lation  coefficients  upon  any  input  data.  This  means 
that  some  coefficients  are  point-biserial  r's  and  some 
are  phi  coefficients.  Standard  corrections  were  ap¬ 
plied  to  each  kind  of  coefficient  to  improve  it  as  an 
estimate  of ’the  Pearson  r.  Thus,  the  coefficients 
reported  in  Table  3  are  all  Pearson-r  coefficients  or 
estimates  of  the  Pearson  r. 


The  correlation  matrix  presented  in  Table  3  is 
typical  of  those  obtained  from  intellectual  measures, 
with  some  exceptions.  The  introduction  of  motivation 
measures  (variables  20,  48,  49,  and  51)  and  experi¬ 
ence  measures  (variables  55,  56,  57,  and  58)  resulted 
In  a  large  number  of  negative  coefficients  not  generally 
found  among  measures  of  intelligence.  It  will  be  seen 
that  when  only  the  intellectual  measures  are  consid¬ 
ered,  however,  there  is  still  a  considerable  number 
of  negative  coefficients,  some  as  largeas  -.13.  These 
coefficients  are  pointed  out,  since  they  will  influence 
the  positive-manifold  requirements  of  the  factor  ma¬ 
trix,  discussed  in  the  next  section. 


Before  discussing  the  factor  analysis,  it  is  ap¬ 
propriate  to  discuss  the  five  variables  not  to  be  factor 
analyzed.  These  variables,  numbers  50,  55,  56,  57, 
and  58,  were  included  in  the  correlational  analysis  to 
determine  their  relationships  to  particular  intellectual 
test  scores.  Examining  the  correlations  with  variable 
50,  sex,  it  can  be  seen  that  15  variables  correlate 
with  sex  at  the  .05  level  of  significance,  and  that  eight 
variables  correlate  at  the  .01  level.  Of  these  eight 
variables,  two  are  such  that  boys  tend  to  have  higher 
scores.  They  are  variables  30,  Series  Relations, 
and  55,  Mathematics  Experience.  "TinKeTeinalnTng 
six  variables,  girls  tend  to  perform  better.  It  is 
interesting  that  most  of  these  variables  are  tests  like 
clerical-  and  perceptual -speed  tests.  Girls  excel  on 
variables  9,  Correct  Spelling;  11,  Derivations;  16, 
Jumbled  Words;  20,  perceptual  Speed;  3b,  Symbol 
idenillles;  $6,  Typing  Experience;  and  57,  Shorthand 
Experience.  All  of  the  significant  correlations  with 
sex  are  reasonable. 
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Table  3 


Correlation  Matrix 


o£  58  Variables  (N  =  ZZ5)^ 


VarUbls 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

1. 

Abbrsvlstions 

32 

33 

40 

14 

36 

35 

38 

31 

36 

27 

17 

24 

14 

26 

30 

36 

27 

19 

-02 

38 

36 

33 

11 

2. 

Bsft  Lsttsr  Sst 

32 

50 

44 

17 

21 

40 

46 

26 

42 

28 

17 

20 

33 

27 

28 

35 

37 

23 

05 

45 

45 

48 

27 

3. 

Bsst  Numbsr  Class 

33 

50 

65 

16 

28 

53 

66 

45 

57 

33 

21 

23 

25 

30 

38 

34 

40 

31 

10 

62 

62 

57 

39 

4. 

Bsst  Numbsr  Pairs 

40 

44 

65 

20 

28 

54 

66 

41 

51 

21 

16 

22 

28 

26 

50 

46 

41 

18 

-03 

59 

64 

69 

19 

5. 

Camouflagsd  Words 

14 

17 

16 

20 

10 

20 

22 

24 

19 

13 

30 

07 

-10 

15 

18 

12 

12 

14 

13 

11 

23 

26 

10 

6. 

Clrcla  Rsasonlng 

36 

21 

28 

28 

10 

25 

33 

28 

28 

19 

21 

19 

16 

16 

38 

20 

31 

09 

02 

22 

33 

33 

-08 

7. 

Corrset  Lsttsr  Ordsrs 

35 

40 

53 

54 

20 

25 

59 

36 

46 

28 

19 

13 

23 

19 

46 

40 

34 

17 

-02 

42 

51 

53 

24 

8. 

Correct  Number  Series 

38 

46 

66 

66 

22 

33 

59 

41 

49 

32 

22 

23 

28 

36 

59 

43 

47 

29 

08 

54 

62 

58 

33 

9. 

Correct  Spelling 

31 

26 

4& 

41 

24 

28 

36 

41 

35 

26 

44 

21 

33 

19 

45 

25 

13 

18 

03 

40 

39 

39 

26 

10. 

Decoding 

36 

42 

57 

51 

19 

28 

46 

49 

35 

26 

17 

17 

15 

25 

41 

36 

27 

18 

02 

45 

53 

53 

22 

U. 

Derivations 

27 

28 

33 

21 

13 

19 

28 

32 

26 

26 

21 

22 

32 

34 

49 

25 

25 

33 

20 

33 

34 

28 

33 

12. 

Disemvowelled  Words 

17 

17 

21 

16 

30 

21 

19 

22 

44 

17 

21 

16 

14 

19 

35 

13 

19 

26 

05 

18 

16 

23 

17 

13. 

Familiar  Letter  Combinations 

24 

20 

23 

22 

07 

19 

13 

23 

21 

17 

22 

16 

25 

18 

28 

21 

1-5 

10 

11 

21 

20 

28 

11 

14. 

Form  Reasoning 

14 

33 

25 

28 

-io 

18 

23 

28 

33 

15 

32 

14 

25 

23 

31 

29 

24 

19 

13 

24 

29 

24 

29 

IS. 

Identical  Forms 

26 

27 

30 

28 

15 

16 

19 

36 

19 

25 

34 

19 

18 

23 

26 

25 

20 

33 

21 

28 

34 

30 

16. 

Jumbled  Words 

30 

28 

38 

50 

18 

38 

46 

59 

45 

41 

49 

35 

28 

31 

26 

41 

44 

33 

19 

39 

50 

39 

17. 

Letter  Problems 

36 

35 

34 

46 

12 

20 

40 

43 

25 

36 

25 

13 

21 

29 

25 

41 

36 

34 

03 

26 

18. 

Letter  Triangle 

27 

37 

40 

41 

12 

31 

34 

47 

13 

27 

25 

19 

15 

24 

20 

44 

36 

20 

-04 

36 

32 

19. 

Letter  U 

19 

23 

31 

18 

14 

09 

17 

29 

18 

18 

33 

26 

10 

19 

33 

33 

34 

20 

38 

26 

20. 

Marking  Speed 

-02 

05 

10 

-03 

13 

02 

•  02 

08 

03 

02 

20 

05 

11 

13 

21 

19 

03 

-04 

38 

01 

1 1 

04 

21. 

Number  Classification 

38 

45 

62 

59 

11 

22 

42 

54 

40 

45 

33 

18 

21 

24 

28 

39 

26 

36 

26 

01 

54 

53 

30 

22. 

Number  Grouping 

36 

45 

62 

64 

23 

33 

51 

62 

39 

53 

34 

18 

20 

29 

34 

50 

42 

32 

28 

11 

54 

58 

20 

23. 

Number-Croup  Naming 

33 

48 

57 

69 

26 

33 

53 

58 

39 

53 

28 

23 

28 

24 

30 

39 

30 

44 

26 

04 

53 

58 

18 

24. 

Numsrlcal  Operations 

11 

27 

39 

19 

10 

-08 

24 

33 

26 

22 

33^ 

17 

11 

29 

19 

41 

10 

19 

36 

33 

30 

20 

18 

2S. 

Operations  Sequence 

28 

51 

59 

52 

19 

28 

41 

58 

37 

45 

33 

29 

18 

34 

31 

46 

38 

44 

35 

09 

53 

50 

55 

35 

2fc. 

-Perceptual  Speed 

21 

27 

33 

26 

10 

15 

20 

34 

14 

22 

43 

08 

20 

38 

56' 

36 

20 

22 

21 

10 

28 

28 

29 

15 

27. 

Related  Words  1 

28 

43 

45 

45 

09 

25 

44 

43 

35 

39 

41 

19 

18 

34 

23 

46 

36 

38 

16 

.06 

44 

43 

39 

19 

28. 

S  Teat 

•  02 

13 

06 

12 

08 

07 

12 

19 

07 

12 

07 

09 

•09 

00 

17 

14 

15 

06 

09 

01 

17 

16 

14 

08 

29. 

Seeing  Trends  II 

30 

33 

35 

49 

24 

25 

57 

48 

32 

39 

19 

13 

20 

29 

22 

45 

40 

38 

06 

-05 

34 

42 

40 

13 

30. 

Series  Relations 

27 

36 

50 

47 

15 

27 

50 

50 

35 

44 

20 

26 

10 

20 

21 

48 

32 

37 

21 

07 

38 

53 

45 

30 

31. 

Sign  Changes 

21 

37 

50 

37 

18 

12 

28 

45 

26 

35 

41 

17 

13 

30 

36 

41 

21 

28 

47 

34 

38 

41 

42 

56 

32. 

Sign  Changes  II 

26 

30 

53 

49 

10 

19 

36 

45 

28 

43 

19 

17 

16 

14 

26 

34 

32 

36 

18 

00 

40 

41 

60 

27 

33. 

Similar  Pairs 

36 

42 

43 

50 

08 

30 

42 

48 

34 

33 

30 

22 

22 

33 

21 

53 

41 

47 

19 

-ll 

36 

41 

36 

11 

34. 

Sound  Grouping 

33 

41 

48 

49 

29 

29 

43 

50 

51 

42 

23 

42 

18 

23 

23 

35 

35 

28 

13 

-01 

38 

42 

45 

11 

35. 

Symbol  Grouping. 

29 

38 

54 

44 

06 

30 

41 

56 

22 

40 

29 

13 

18 

27 

26 

39 

36 

36 

26 

01 

45 

48 

45 

21 

36. 

Symbol  Identities 

24 

24 

30 

34 

09 

08 

30 

41 

42 

30 

47 

20 

09 

37 

50 

49 

32 

21 

51 

27 

33 

40 

31 

41 

37. 

Symbol  Manipulation 

25 

38 

68 

48 

19 

33 

40 

50 

26 

38 

31 

15 

16 

30 

19 

38 

29 

45 

14 

-11 

46 

S3 

53 

06 

38. 

Symbol  Reasoning 

33 

42 

62 

49 

13 

36 

56 

60 

33 

44 

30 

15 

18 

22 

28 

11 

39 

42 

16 

-02 

43 

51 

53 

17 

39. 

Typing  Errors 

13 

29 

29 

37 

14 

16 

28 

28 

20 

30 

20 

15 

11 

32 

15 

29 

23 

26 

17 

01 

33 

34 

28 

13 

40. 

Varied  Symbols 

03 

11 

•01 

09 

12 

07 

12 

17 

09 

16 

26 

27 

02 

09 

19 

19 

07 

11 

12 

-02 

06 

13 

11 

13 

41. 

Way-Out  Numbers 

25 

24 

59 

46 

08 

14 

42 

53 

23 

29 

25 

13 

10 

35 

30 

36 

26 

33 

20 

11 

35 

41 

49 

41 

42. 

Word  Changes 

48 

39 

49 

47 

19 

34 

49 

56 

29 

44 

29 

34 

20 

30 

33 

58 

34 

S3 

32 

05 

42 

51 

66 

29 

43. 

Word  Choice 

27 

34 

39 

41 

02 

34 

36 

46 

22 

41 

32 

12 

14 

29 

17 

41 

40 

30 

15 

00 

37 

47 

44 

17 

44. 

Word  Combinations 

29 

35 

35 

36 

23 

30 

48 

45 

40 

35 

23 

37 

23 

27 

22 

45 

31 

28 

IS 

02 

29 

39 

44 

20 

45. 

Word  Patterns 

25 

20 

33 

30 

25 

28 

30 

36 

26 

26 

29 

27 

18 

23 

27 

38 

31 

27 

24 

08 

31 

40 

43 

15 

46. 

Word  Relations 

32 

37 

48 

54 

20 

29 

49 

57 

35 

40 

39 

25 

27 

27 

39 

59 

40 

43 

14 

03 

43 

47 

49 

25 

47, 

Word  Transformation 

31 

36 

54 

40 

33 

33 

41 

42 

49 

39 

33 

46 

23 

11 

24 

44 

23 

32 

19 

09 

42 

44 

40 

27 

4t. 

Rating  Test  Liking 

15 

20 

23 

21 

19 

14 

19 

18 

21 

15 

15 

24 

09 

•  01 

10 

20 

19 

24 

14 

06 

22 

28 

21 

05 

49. 

Rating  Booklet  Effort 

05 

•09 

•  08 

•02 

03 

02 

-05 

-06 

06 

.06 

11 

03 

-05 

03 

-06 

-06 

•  03 

08 

01 

-03 

-02 

-01 

•  13 

•  01 

50. 

Sex 

•  It 

-08 

09 

-06 

-09 

-08 

•  02 

00 

-18 

01 

-31 

•09 

•  11 

-11 

•  10 

-19 

•  15 

-01  , 

•  03 

.01 

.04  . 

.08 

-02 

10 

51. 

Prediction- Achievement  Discrepancy 

15 

11 

10 

18 

•  18 

02 

14 

09 

05 

13 

•  02 

-10 

-07 

14 

01 

13 

08 

03 

11 

-05 

08 

17 

10 

10 

52. 

ITSO  General  Vocabulary 

36 

41 

54 

56 

11 

30 

65 

56 

49 

43 

16 

20 

16 

20 

25 

36 

45 

34 

11 

-10 

36 

48 

52 

06 

53. 

PSAT  Verbal 

30 

25 

39 

40 

39 

23 

51 

52 

47 

46 

14 

20 

13 

15 

27 

56 

28 

25 

11 

.03 

26 

42 

54. 

SCAT  Verbal 

27 

38 

30 

41 

17 

11 

47 

47 

44 

29 

08 

30 

07 

32 

15 

34 

21 

23 

•  05 

-19 

31 

37 

40 

06 

55. 

Mathematics  Experience 

25 

31 

52 

50 

11 

27 

46 

56 

28 

41 

04 

13 

05 

16 

16 

33 

27 

28 

IS 

04 

39 

51 

57 

18 

56. 

Typing  Experience 

-02 

-20 

-28 

-13 

06 

00 

-14 

-09 

-01 

-19 

12 

10 

-04 

■  10 

00 

06 

-13 

-04 

04 

15 

.09 

•  15 

•18 

07 

57. 

Storthind  Experience 

-08 

-03 

-16 

-09 

11 

-07 

-12 

-15 

00 

-10 

14 

15 

09 

-10 

06 

00 

-09 

-05 

01 

13 

01 

-05 

-06 

00 

58. 

Test  Administrator 

-07 

-07 

04 

12 

-24 

11 

16 

09 

01 

13 

09 

-09 

-02 

09 

-06 

26 

04 

06  . 

-04 

-03 

02 

13 

-04  . 

■  02 

*0«cimal  poiau  otnUua. 

aampU  o(  Z2S  sCudaata  daUrmiaad  Um  coafdclaau  for  varlabloo  1  through  50  and  varlabloi  55  through  58.  Coefficients 


for  variables  51,  52  ,  53,  and  54  were  detormined  reepoctively  from  aubeamplee  of  165,  182,  109,  and  107  an  school  records  wars' incomplete. 


Variable  55,  Mathematics  Ebcperience,  was  included 
in  the  correlational  analysis  to  determine  which,  if  any, 
of  the  aptitude-test  scores  could  possibly  depend  upon 
the  amount  of  mathematical  experience  K  had  had.  Fifty 
of  the  variables  correlate  at  the  .  05  level  of  significance 
with  the  index  of  mathematics  experience.  This  large 
number  of  significant  correlation  coefficients  could  be 
due  to  two  things:  (1)  experience  in  mathematics  courses 
trains  individuals  at  proficiency  in  tasks  measured  by 
symbolic  tests,  or  (2)  individuals  who  get  more  mathe¬ 
matics  training  are  relatively  high  in  abilities  measured 
by  the  tests,  is  probable  that  both  explanations  under¬ 
lie  the  coefficients,  but  the  second  would  have  greater 
practical  significance.  If  the  second  explanation  is 
correct,  the  tests  correlating  with  mathematics  experi¬ 
ence  show  promise  as  selectors  of  mathematics  students. 


To  determine  whether  or  not  scores  on  test  39, 
Typing  Errors,  are  related  to  experience  in  typing,  the 


number  of  semesters  of  formal  high-school  typing 
courses  taken  was  entered  into  the  correlational  analy¬ 
sis.  The  two  variables,  numbers  39  and  56, correlate 
,  03,  signifying  that  there  is  little  or  no  relationship 
between  them.  It  is  interesting  to  note  that  Typing 
Experience  is  correlated  positively  and  significantly 
only  with  Disemvowelled  Words  (a  CSU  test),  in  which 
word  spelling  and  recognition  are  important,  with 
Marking  Speed,  a  measure  of  finger  speed,  and  with 
Shorthand  Experience.  It  is  significantly  negatively 
correlated  with  most  of  the  numerical  tests  and  with 
all  three  verbal-comprehension  tests. 

Variable  57,  Shorthand  Experience,  was  analyzed 
to  determine  its  relationship  with  variable  1,  Abbrevi¬ 
ations.  The  two  variables  under  consideration  corre¬ 
late  negatively,  but  not  significantly.  Shorthand  Expe¬ 
rience  correlates  positively  with  word  and  speed  tests 
and  with  Typing  Experience,  but  correlates  negatively 
with  many  numerical  tests  and  all  three  verbalvcompre- 
henaion  tests. 


IS 


Table  3  (Continued) 


25 

26 

27 

28 

29 

30 

31 

32 

33 

34 

35 

36 

37 

38 

39 

40 

41 

42 

43 

44 

45 

46 

47 

48 

49 

SO 

51 

52 

53 

54 

55 

56 

57 

58 

28 

21 

28 

.02 

30 

27 

21 

26 

36 

33 

29 

24 

25 

33 

13 

03 

25 

48 

27 

29 

25 

32 

31 

15 

05 

.11 

15 

36 

30 

27 

25 

.02 

-08 

-07 

51 

27 

43 

13 

33 

36 

37 

30 

42 

41 

38 

24 

38 

42 

29 

11 

24 

39 

34 

35 

20 

37 

36 

20 

-09 

.08 

n 

41 

25 

38 

31 

.20 

.03 

-07 

59 

33 

45 

06 

35 

50 

50 

S3 

43 

48 

54 

30 

68 

62 

29 

-01 

59 

49 

39 

35 

33 

48 

54 

23 

.08 

09 

10 

54 

39 

30 

52 

-28 

-16 

04 

52 

26 

45 

12 

49 

47 

37 

49 

50 

49 

44 

34 

48 

49 

37 

09 

46 

47 

41 

36 

30 

54 

40 

21 

-02 

.06 

18 

56 

40 

41 

50 

-13 

.09 

12 

19 

10 

09 

08 

24 

15 

18 

10 

08 

29 

06 

09 

19 

13 

14 

12 

08 

.19 

■02 

23 

25 

20 

33 

19 

03 

-09 

-18 

11 

39 

17 

11 

06 

11 

.24 

28 

15 

25 

07 

*25 

27 

12 

19 

30 

29 

30 

08 

33 

36 

16 

07 

14 

34 

34 

30 

28 

29 

33 

14 

02 

.08 

02 

30 

23 

11 

27 

00 

.07 

11 

41 

20 

44 

12 

57 

50 

28 

36 

42 

43 

41 

30 

40 

56 

28 

12 

48 

49 

36 

42 

30 

49 

41 

19 

.05 

.02 

14 

65 

51 

47 

48 

-14 

•  12 

16 

58 

34 

43 

19 

48 

50 

45 

45. 

^8 

50 

56 

41 

50 

60 

28 

17 

53 

56 

46 

45 

36 

57 

42 

18 

.06 

00 

09 

56 

52 

47 

56 

-09 

.15 

09 

37 

14 

35 

07 

32 

35 

26 

28 

34 

51 

22 

42 

26 

33 

20 

09 

23 

29 

22 

40 

26 

35 

49 

21 

06 

-18 

05 

49 

47 

44 

28 

.01 

00 

01 

45 

22 

39 

12 

39 

44 

35 

43 

33 

42 

40 

30 

38 

44 

30 

16 

29 

44 

41 

35 

26 

40 

39 

15 

-06 

01 

13 

43 

46 

29 

41 

.19 

-10 

13 

3} 

43 

41 

07 

19 

20 

41 

19 

30 

23 

29 

47 

31 

30 

20 

26 

25 

29 

32 

23 

29 

39 

33 

15 

11 

.31 

-02 

16 

14 

08 

04 

12 

14 

09 

29 

08 

19 

09 

13 

26 

17 

17 

22 

42 

13 

20 

15 

15 

15 

27 

13 

34 

12 

37 

27 

25 

46 

24 

03 

-09 

-10 

20 

20 

30 

13 

10 

15 

-09 

18 

20 

18 

-09 

20 

10 

13 

16 

22 

18 

18 

09 

16 

18 

11 

02 

10 

20 

14 

23 

18 

27 

23 

09 

-05 

-11 

-07 

16 

13 

07 

05 

.04 

09 

-02 

34 

38 

34 

00 

29 

20 

30 

14 

33 

23 

27 

3? 

30 

22 

32 

09 

35 

30 

29 

27 

23 

27 

11 

-01 

03 

-11 

14 

20 

15 

32 

16 

-10 

.10 

09 

31 

56 

23 

17 

22 

21 

36 

26 

21 

23 

26 

50 

19 

28 

15 

19 

30 

33 

17 

22 

27 

39 

24 

10 

-06 

.10 

01 

25 

27 

15- 

18 

00 

06 

.06 

46 

36 

46 

14 

45 

48 

41 

34 

53 

35 

39 

49 

38 

41 

29 

19 

36 

56 

41 

45 

36 

59 

44 

20 

-06 

-19 

13 

36 

56 

34 

33 

06 

00 

26 

38 

20 

36 

1-5 

40 

32 

21 

32 

41 

35 

38 

32 

29 

39 

23 

07 

26 

34 

40 

31 

31 

40 

23 

19 

-03 

-15 

08 

45 

28 

21 

27 

-13 

-09 

04 

44 

22 

38 

06 

38 

37 

28 

38 

47 

28 

36 

21 

45 

42 

26 

11 

33 

53 

30 

28 

27 

43 

32 

24 

08 

.01 

03 

34 

25 

23 

28 

.04 

-05 

06 

35 

21 

16 

09 

08 

21 

47 

18 

19 

13 

26 

51 

14 

18 

17 

12 

20 

32 

15 

15 

24 

14 

19 

14 

01 

-03 

11 

11 

11 

-05 

15 

04 

01 

.04 

09 

10 

.06 

01 

-05 

07 

34 

00 

-11 

-01 

01 

27 

-11 

.02 

01 

.02 

14 

05 

00 

02 

08 

.03 

09 

06 

-03 

.01 

.05 

-10 

-03 

-19 

04 

15 

13 

-03 

53 

28 

44 

17 

34 

38 

38 

40 

36 

38 

45 

33 

46 

43 

33 

08 

35 

42 

37 

29 

31 

43 

42 

22 

.02 

.04 

08 

36 

26 

31 

39 

-09 

01 

02 

50 

28 

43 

16 

42 

53 

41 

41 

41 

42 

48 

40 

53 

51 

34 

13 

41 

51 

47 

39 

40 

47 

44 

28 

-01 

.08 

17 

48 

42 

37 

51 

-is 

-05 

13 

55 

29 

39 

14 

40 

45 

42 

60 

38 

45 

45 

31 

53 

53 

28 

11 

49 

66 

44 

44 

43 

49 

40 

21 

-13 

.02 

10 

52 

31 

40 

57 

-18 

.06 

-04 

35 

15- 

19 

08 

13 

30 

56 

27 

11 

11 

21 

41 

06 

17 

13 

13 

41 

29 

17 

20 

15 

25 

27 

05 

-01 

10 

10 

06 

04 

06 

18 

07 

00 

-02 

26 

45 

16 

30 

55 

50 

49 

42 

47 

45 

37 

50 

55 

40 

21 

38 

47 

39 

33 

41 

49 

37 

29 

.06 

13 

07 

37 

28 

37 

5i 

-18 

-12 

-01 

26 

31 

16 

23 

15 

29 

19 

27 

19 

37 

32 

19 

26 

IS 

12 

19 

23 

23 

18’ 

24 

34 

19 

03 

-03 

-18 

08 

22 

26 

24 

16 

.02 

.00 

06 

a 

31 

04 

35 

36 

29 

35 

SO 

40 

48 

30 

49 

41 

32 

14 

33 

44 

40 

25 

19 

59 

27 

22 

02 

.14 

.02 

43 

26 

33 

22 

-12 

.02 

.12 

16 

16 

04 

12 

07 

06 

16 

04 

08 

18 

17 

00 

11 

10 

22 

10 

04 

11 

08 

^16 

«09 

09 

09 

-13 

'03 

-09 

16 

-09 

01 

30 

08 

02 

-15 

30 

23 

35 

12. 

41 

22 

28 

44 

38 

31 

22 

44 

47 

22 

14 

35 

49 

31 

36 

32 

52 

36 

22 

02 

-09 

07 

54 

55 

46 

35 

-14 

-15 

13 

55 

15 

36 

07 

41 

38 

36 

41 

39 

47 

24 

45 

51 

31 

11 

48 

54 

32 

35 

27 

48 

38 

27 

00 

18 

10 

42 

32 

44 

46 

-17 

-14 

11 

50 

29 

29 

06 

22 

38 

33 

24 

17 

33 

51 

16 

29 

21 

16 

44 

40 

20 

17 

27 

34 

26 

12 

.08 

07 

06 

17 

12 

02 

27 

-01 

05 

.05 

49 

19 

35 

16 

28 

36 

33 

28 

41 

42 

25 

39 

40 

23 

18 

39 

43 

33 

26 

26 

41 

33 

14 

.04 

06 

-02 

38 

19 

30 

45 

-19 

.20 

-03 

42 

27 

50 

04 

44 

41 

24 

28 

40 

47 

34 

51 

48 

35 

09 

34 

49 

41 

37 

26 

55 

33 

21 

01 

-15 

07 

50 

33 

41 

31 

.06 

-03 

10 

47 

19 

40 

08 

38 

39 

17 

41 

40 

34 

26 

40 

44 

25 

IS 

26 

43 

30 

44 

33 

47 

47 

22 

-09 

-12 

-12 

60 

52 

56 

37 

-10 

.08 

02 

45 

37 

48 

18 

31 

47 

33 

42 

47 

34 

27 

43 

44 

26 

09 

36 

45 

39 

32 

34 

43 

27 

16 

-04 

-02 

-05 

44 

23 

19 

43 

-08 

-09 

06 

37 

32 

30 

17 

22 

24 

51 

25 

34 

26 

27 

14 

31 

19 

18 

33 

36 

26 

20 

26 

35 

24 

09 

00 

-25 

13 

35 

26 

11 

23 

10 

08 

08 

50 

19 

49 

00 

44 

45 

16 

39 

51 

40 

43 

14 

56 

30 

03 

30 

48 

44 

26 

39 

45 

36 

18 

01 

-03 

01 

43 

44 

50 

33 

-25 

-10 

05 

55 

28 

41 

11 

47 

51 

29 

40 

48 

44 

44 

31 

56 

32 

11 

49 

40 

38 

36 

31 

54 

33 

20 

-03 

05 

08 

57 

52 

42 

53 

-22 

-11 

16 

40 

15 

32 

10 

22 

31 

21 

23 

35 

25 

26 

19 

30 

32 

11 

20 

23 

19 

23 

17 

31 

26 

08 

.01 

04 

07 

27. 

24 

19 

26 

03 

04 

00 

21 

12 

14 

22 

14 

11 

16 

18 

09 

15 

09 

18 

03 

11 

11 

03 

20 

15 

13 

21 

26 

20 

05 

-06 

03 

-13 

10 

00 

05 

16 

04 

11 

01 

38 

19 

33 

10 

35 

48 

44 

39 

34 

26 

36 

33 

30 

49 

20 

03 

49 

25 

25 

14 

40 

24 

14 

-04 

11 

04 

35 

30 

30 

33 

-14 

■09 

18 

47 

23 

44 

04 

49 

54 

40 

43 

49 

43 

45 

38 

48 

40 

23 

20 

49 

38 

40 

34 

58 

49 

19 

01 

14 

04 

43 

52 

34 

40 

.05 

-09 

01 

39 

23 

40 

11 

31 

32 

20 

33 

41 

30 

39 

26 

44 

38 

19 

15 

25 

38 

32 

36 

44 

27 

16 

-03 

10 

02 

37 

24 

35 

31 

.10 

-09 

10 

33 

18 

25 

08 

36 

35 

17 

26 

37 

44 

32 

20 

26 

36 

23 

13 

25 

40 

32 

31 

43 

47 

22 

06 

14 

00 

39 

40 

32 

32 

.08 

00 

03 

41 

24 

19 

16 

32 

27 

27 

26 

26 

33 

34 

26 

39 

31 

17 

21 

14 

34 

36 

31 

34 

41 

30 

.01 

03 

.02 

30 

27 

12 

35 

00 

-02 

-09 

49 

34 

59 

09 

52 

48 

34 

41 

55 

47 

43 

35 

45 

54 

31 

26 

40 

58 

44 

43 

34 

40 

25 

02 

06 

-05 

46 

32 

31 

38 

•  13 

-11 

16 

37 

19 

27 

09 

36 

38 

26 

33 

33 

47 

27 

24 

36 

33 

26 

20 

24 

49 

27 

47 

41 

40 

25 

05 

05 

00 

43 

44 

45 

31 

-02 

01 

05 

29 

03 

22 

09 

22 

27 

12 

14 

21 

22 

16 

09 

18 

20 

08 

05 

14 

19 

16 

22 

30 

25 

25 

35 

04 

-12 

21 

11 

05 

22 

.04 

00 

05 

.06 

.03 

02 

.13 

02 

00 

.08 

•04 

01 

.09 

'04 

00 

01 

.03 

-01 

.06 

.04 

01 

-03 

06 

.01 

02 

05 

35 

09 

-03 

.02 

04 

.12 

-07 

09 

.01 

14 

13 

-18 

.14 

.03 

-09 

18 

07 

06 

.15 

.12 

-02 

-2$ 

-03 

05 

-04 

-03 

11 

-14 

-10 

-14 

03 

.06 

.05 

.04 

.09 

.03 

.03 

.06 

06 

30 

-35 

.35 

-01 

07 

08 

.02 

.09 

07 

10 

06 

.02 

07 

.12 

-05 

13 

01 

08 

07 

-13 

04 

04 

02 

00 

-02 

.05 

00 

.12 

.03 

03 

11 

23 

18 

17 

.09 

.06 

12 

37 

22 

43 

16 

54 

42 

17 

38 

50 

60 

44 

35 

43 

57 

27 

10 

35 

43 

3? 

39 

30 

46 

43 

21 

.02 

03 

11 

71 

63 

57 

-30 

.22 

12 

28 

26 

26 

.09 

55 

32 

12 

19 

33 

52 

23. 

26 

44 

52 

24 

00 

30 

52 

24 

40 

27 

32 

44 

11 

04 

06 

23 

71 

66 

50 

.18 

.19 

06 

37 

24 

33 

01 

46 

44 

02 

30 

41 

56 

19 

11 

50 

42 

19 

05 

30 

34 

35 

32 

12 

31 

45 

05 

-12 

06 

18 

63 

66 

37 

-39 

.33 

.02 

51 

16 

22 

30 

35 

46 

27 

45 

31 

37 

43 

23 

33 

53 

26 

16 

33 

40 

31 

32 

35 

38 

31 

22 

.07 

30 

17 

57 

50 

37 

-14 

.24 

-01 

.18 

-02 

.12 

08 

-14 

-01 

.19 

-06 

.10 

.08 

10 

-25 

.22 

03 

04 

.14 

.05 

-10 

-08 

00 

.13 

.02 

.04 

09 

.35 

-09 

.30 

-18 

-39 

-14 

48 

-09 

.12 

00 

.02 

02 

-15 

-14 

05 

.20 

-03 

.08 

.09 

08 

-10 

-11 

04 

11 

-09 

-09 

-09 

00 

-02 

-11 

01 

00 

.01 

.35 

.06 

-22 

-19 

.33 

.24 

48 

-05 

-01 

06 

12 

.15 

13 

11 

-05 

-03 

10 

02 

06 

08 

05 

16 

00 

01 

18 

01 

10 

03 

.09 

16 

05 

05 

14 

-01 

12 

12 

06 

.02 

.01 

.09 

.05 

Only  three  tests  correlate  at  the  .  01  level  of  sig¬ 
nificance  with  variable  58,  Test  Administrator.  For 
unexplainable  reasons,  the  students  in  the  smaller  group 
tended  to  score  higher  on  test  16,  Jumbled  Words,  and  41 , 
Wav-CXit  Numbers.  Students  in  the  larger  group  tended 
to  score  higher  on  test  5,  Camouflaged  Words.  In  gen¬ 
eral,  however,  the  correlations  with  variable  58  were 
low. 


The  five  variables  just  discussed  are  either  not 
appropriate  for  factor  analysis  or  would  have  confused 
the  hypothesized  factor  structure  due  to  their  experi¬ 
ential  natures.  The  high  correlations  across  the  board 
with  Mathematics  Experience  might  generate  a  g-type 
factor,  and,  similarly,  other  experience  indexes,  cor¬ 
related  with  composites  of  tests,  might  generate  group- 
factors,  thus  militating  against  a  clear  test  of  the  hypo¬ 
theses  derived  from  the  structure-of-intellect  model. 
Discussion  of  the  five  variables  ends  here. 


Factor  Analysis 

Thp  correlation  matrix  of  all  variables  except  num-  * 
bers  50,  55,  56,  57,  and  58  was  submitted  to  an  iterative 
communality-estimation  program  for  estimates  of  com- 
munalities  to  be  inserted  into  the  principal-diagonal  cells 
of  the  correlation  matrix  for  factor  analysis.  Starting 
with  the  highest-in-column  entries,  the  iterative  pro¬ 
gram  extracted  19  factors  (the  hypothesized  number  of 
factors),  computed  the  new  communalities,  replaced  the 
diagonal  entries  with  the  new  communalities,  and  re- 
extracted  factors.  This  procedure  was  iterated  eight 
times  until  the  communality  estimates  stabilized  from 
one  iteration  cycle  to  the  next. 

The  iterated,  stabilized  communality  estiknates 
were  put  into  the  diagonal  cells  of  the  correlation  matrix 
and  the  matrix  was  submitted  to  the  BIMD  17  program 
at  the  Western  Data  Processing  Center.  BIMD  17  ex¬ 
tracts  principal-axes  factors  until  the  eigenvalues  be¬ 
come  negative,  at  which  point  extractions  are  stopped. 
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Table  4 


Unrotated  Factor  Matrix* 


V»r. 

A 

B 

C 

D 

E 

F 

G 

H 

I 

J 

K 

L 

M 

N 

O 

P 

Q 

R 

S 

T 

h2 

1. 

50 

05 

-06 

-01 

-11 

08 

24 

18 

10 

14 

19 

00 

■  06 

-08 

.10 

18 

07 

■  08 

-10 

-11 

52 

2. 

59 

.02 

11 

.02 

03 

-03 

-13 

20 

03 

-03 

^05 

-21 

■  06 

21 

.11 

25 

07 

■  08 

-03 

00 

61 

3. 

77 

.05 

21 

00 

30 

07 

-02 

19 

-20 

-06 

04 

16 

08 

-05 

-02 

09 

-04 

11 

01 

02 

87 

4. 

75 

11 

19 

02 

12 

-01 

08 

10 

13 

-03 

12 

.02 

-12 

07 

■  06 

-21 

-08 

09 

00 

-04 

76 

5. 

28 

05 

.42 

.08 

22 

-11 

14 

-03 

-19 

-04 

-16 

-0.6 

-05 

19 

-14 

-06 

06 

06 

06 

-06 

49 

6. 

42 

11 

-07 

-22 

-15 

03 

14 

14 

04 

19 

-02 

11 

08 

-08 

08 

03 

-15 

-15 

-10 

-01 

43 

•  7, 

69 

18 

07 

13 

07 

-02 

06 

-17 

11 

-09 

01 

08 

-03 

10 

07 

11 

08 

-02 

.05 

-10 

64 

6. 

80 

00 

08 

07 

08 

.07 

07 

-07 

-01 

01 

01 

05 

06 

03 

05 

03 

-07 

06 

-04 

01 

69 

9. 

56 

08 

-35 

15 

05 

09 

-20 

18 

11 

-12 

17 

13 

04 

.08 

02 

-11 

-05 

02 

-02 

-05  ■ 

65 

10. 

64 

04 

04 

00 

15 

-07 

08 

08 

13 

06 

-03 

18 

-12 

06 

-15 

01 

03 

-13 

07 

16 

60 

11. 

46 

-37 

.10 

-06 

-30 

03 

-10 

01 

-09 

-07 

-06 

24 

-09 

-05 

-03 

0.7 

14 

-03 

-06 

-03 

58 

12. 

37 

-04 

-54 

-14 

04 

03 

-24 

01 

04 

09 

01 

-11 

03 

-14 

01 

04 

-09 

02 

04 

-05 

57 

13. 

30 

-06 

-07 

-07 

-14 

03 

01 

20 

.12 

17 

21 

03 

12 

21 

-01 

-07 

01 

03 

-08 

01 

32 

14. 

43 

-19 

10 

18 

-34 

14 

-27 

14 

-01 

02 

03 

-16 

04 

09 

23 

-13 

02 

-0.4 

-02 

03 

60 

15. 

45 

-32 

.10 

12 

-16 

-34 

17 

04 

-23 

-05 

10 

-16 

-08 

-10 

.01 

00 

-09 

-08 

01 

00 

62 

16. 

69 

-12 

-15 

11 

-21 

12 

00 

-21 

10 

19 

-12 

23 

02 

00 

-04 

05 

-12 

11 

01 

04 

77 

17. 

55 

-01 

06 

-01 

-19 

-03 

17 

00 

27 

-08 

-01 

-09 

21 

11 

-07 

02 

08 

08 

-01 

08 

54 

U. 

56 

01 

12 

^21 

-10 

16 

06 

-17 

-06 

09 

-06 

-20 

-04 

-05 

-06 

06 

-03 

15 

-16 

08 

56 

19. 

37 

-52 

-12 

08 

05 

08 

15 

09 

n 

-04 

-16 

-17 

18 

-13 

-10 

03 

04 

08 

04 

00 

61 

20. 

07 

-51 

-17 

12 

12 

05 

1 1 

06 

-09 

01 

-06 

02 

20 

15 

06 

-05 

02 

-09 

03 

04 

43 

21. 

65 

-07 

15 

-07 

13 

00 

.06 

21 

02 

-08 

04 

08 

-18 

-03 

-03 

00 

-04 

11 

-05 

-13 

60 

22. 

75 

^01 

08 

00 

09 

-01 

15 

13 

07 

-06 

-12 

09 

-07 

04 

09 

■09 

-03 

-12 

12 

-06 

70 

23. 

75 

03 

12 

.11 

22 

-12 

07 

09 

00 

19. 

•10 

.13 

.07 

-05 

08 

-18 

10 

.03 

02 

-02- 

79 

24. 

36 

-52 

.03 

19 

•  23 

20 

-21 

-16 

00 

02 

06 

08 

-05 

>2 

06 

1'3 

08. 

;i 

-02 

06 

67  , 

25. 

71 

.14 

08 

-14 

13 

-01 

-17 

05 

04 

-05 

-20 

-13 

01 

-05 

-01 

-02 

-03 

■09 

-13 

07 

69 

26. 

43 

-24 

04 

16 

-36 

-29 

05 

13 

-29 

.03 

02 

-03 

-17 

02 

07 

10 

-11 

05 

06 

09 

64 

27. 

62 

01 

15 

-12 

-24 

07 

-20 

-04 

-04 

-12 

05 

07 

-03 

01 

-23 

-02 

07 

-05 

15 

-04 

63 

28. 

16 

-12 

-01 

-12 

04 

-39 

-01 

-11 

22 

-12 

-06 

-02 

-07 

00 

14 

05 

-10 

14 

05 

-06 

35 

29. 

6l 

25 

-03 

09 

-09 

01 

15 

-20 

-01 

-03 

02 

-08 

-07 

23 

08 

-05 

09 

05 

-04 

-07 

61 

30. 

65 

04 

06 

-03 

18 

16 

-05 

-17 

02 

01 

-10 

-05 

03 

00 

09 

08 

-17 

-21 

10 

-03 

63 

31. 

51 

-55 

05 

09 

20 

06 

00 

-03 

-05 

02 

-01 

-03 

-02 

05 

-06 

-01 

03 

-08 

-04 

-05 

64 

32. 

57 

-03 

IS 

-16 

22 

-14 

-07 

-06 

03 

08 

IS 

-02 

-03 

-15 

-02 

-08 

09 

08 

-01 

27 

60 

33. 

64 

13 

11 

-05 

-29 

15 

-06 

-06 

04 

-01 

00 

-08 

07 

-01 

-12 

03 

-07 

10 

03 

-10 

61 

34. 

64 

25 

-26 

.02 

07 

-15 

-19 

08 

00 

-05 

lb 

-06 

17 

-02 

.10 

-02 

06 

■06 

03 

05 

68 

35. 

62 

-07 

23 

-16 

-07 

-07 

04 

00 

-01 

-03 

04 

03 

16 

00 

03 

15 

-14 

04 

21 

03 

60 

36. 

51 

-45 

-10 

29 

-12 

-05 

04 

-04 

16 

-16 

04 

-01 

03 

-19 

-06 

-13 

07 

01 

01 

-07 

70 

37. 

66 

25 

20 

-18 

-03 

12 

.05 

10 

-28 

02 

.27 

02 

06 

-11 

04 

-08 

13 

10 

06 

-07 

81 

38. 

67 

13 

18 

00 

02 

.12 

02 

.10 

.11 

-16 

.08 

11 

13 

-07 

01 

02 

-03 

■12 

-28 

00 

70 

39. 

42 

.01 

07 

.02 

-04 

03 

-15 

05 

07 

-07 

-20 

-08 

-07 

13 

-12 

.18 

.20 

-02 

.08 

03 

38 

40. 

19 

•  IS 

-20 

-20 

^07 

-26 

-14 

-25 

13 

06 

-08 

04 

-17 

-03 

-02 

00 

06 

■  07 

■  06 

01 

36 

41. 

56 

-13 

22 

14 

18 

08 

02 

-28 

.18 

-02 

18 

-01 

06 

-05 

13 

-03 

.05 

-02 

.01 

.07 

61 

42. 

72 

-02 

.06 

-05 

01 

15 

13 

.22 

-06 

32 

06 

-16 

-05 

-13 

-06 

01 

04 

-07 

11 

.05 

79 

43. 

56 

04 

14 

.13 

-18 

.02 

.02 

04 

16 

07 

.08 

15 

03 

00 

17 

00 

19 

-06 

10 

08 

52 

44. 

56 

10 

.26 

-06 

-06 

05 

-02 

02 

08 

09 

10 

.02 

02 

16 

12 

05 

-10 

00 

01 

10 

49 

45. 

SO 

-07 

-19 

-22 

-05 

.08 

17 

11 

04 

05 

.13 

01 

02 

.02 

23 

-04 

11 

08 

-05 

■06 

49 

46. 

71 

.01 

00 

-16 

-18 

.02 

.04 

-20 

-06 

01 

13 

06 

■  05 

11 

.08 

.11 

-02 

-05 

03 

-03 

66 

47. 

61 

08 

-38 

.07 

15 

05 

.06 

08 

.05 

08 

00 

09 

-11 

01 

04 

12 

-04 

10 

-02 

.01 

62 

46. 

31 

01 

.20 

-35 

06 

19 

15 

.01 

01 

-34 

04 

.10 

■  06 

02 

10 

02 

01 

■06 

04 

02 

47 

49. 

-02 

02 

-16 

-18 

.12 

39 

16 

.01 

-07 

-36 

10 

.02 

.20 

-08 

08 

.02 

01 

-03 

00 

16 

46 

51. 

09 

03 

17 

42 

-01 

20 

11 

17 

22 

10 

-10 

-07 

-22 

-07 

02 

04 

-06 

02 

01 

03 

43 

52. 

70 

39 

-03 

21 

-01 

-13 

07 

.03 

12 

-20 

12 

-02 

14 

-07 

.01 

06 

04 

02 

-01 

02 

81 

53. 

61 

42 

-29 

44 

-02 

04 

22 

.06 

-16 

-01 

-16 

07 

02 

-02 

-10 

.04 

-01 

01 

00 

13 

97 

54. 

56 

51 

.08 

34 

00 

.05 

.30 

.01 

.08 

02 

.08 

-13 

-05 

-07 

12 

07 

09 

-02 

03 

00 

85 

aDcclmal  polnte  omitted. 


This  procedure  resulted  in  the  extraction  of  34  principal- 
axes  factors.  The  first  19  factors  accounted  for  93,  6 
per  cent  of  the  total  variance  of  the  34-factor  matrix. 
Inclusion  of  the  20th  factor,  which  had  a  range  of  load¬ 
ings  from  -.  13  to  .  27,  as  a  potential  residual  factor, 
accounted  for  94.  4  per  cent  of  the  total  variance.  The 
20-factor  matrix  was  accepted  as  the  principal-axes 
factor  mstrbi  to  be  rotated.  With  20  factors,  the  average 
input-output  communality  discrepancy  was  .  029.  This 
factor  matrix  is  presented  in  Table  4. 

Nineteen  of  the  20  factors  of  the  principal-axes 
matrix  were  analytically  rotated  to  quartimax,  varimax, 
and  equamax  solutions.  Since  all  three  rotational  cr>. 
teria  tended  to  make  each  factor  rotated  at  least  a  strong 
singlet,  and  since  one  residual  factor  was  expected,  the 
20th  principal-axes  factor  was  withheld  from  the  analyti¬ 
cal  rotational  solutions.  The  equamax  solution  appeared 
more  in  line  with  the  hypothesized  factor  structure  be¬ 
cause  of  its  tendency  to  equalize  approximately  the 
variance  on  each  factor  while  seeking  simple  structure. 


The  19  equannax-rotated  factors,  along  with  the  20th 
principal-axes  factor,  were  then  graphically  rotated  to 
orthogonal  simple  structure  and  psychological  meaning¬ 
fulness,  The  rotational  criterion  of  positive  manifold 
was  necessarily  violated  due  to  the  many  negative  rela¬ 
tionships  between  variables.  After  61  rotations,  it  was 
clear  that  most  of  the  hypothesized  factors  would  emerge  , 
but  that  the  criterion  of  simple  structure  would  not 
strictly  be  met  without  further  minor  adjustments  in 
the  rotations. 

The  graphic  solution  was  submitted  to  a  program 
designed  to  rotate  the  loadings  as  closely  as  possible  to 
a  fixed  target  matrix  of  loadings  (Cliff,  1964),  The  con¬ 
struction  of  the  target  matrix  depended  upon  the  intui¬ 
tively  inferred  structure  of  the  empirical  matrix,  sim¬ 
ple  structure,  positive  manifold,  and  the  factor  hypo¬ 
theses.  Four  slight  successive  adjustments  or  revisions 
of  the  target  matrix  effected  considerable  improvement 
in  the  empirical  rotated  matrix  on  all  four  criteria.  The 
principal-axes  matrix  was  then  rotated  to  the  fifth  tarr 
get  matrix  so  that  rounding  and  graphic  errors  would 
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Tabu  S 


Routed  Factor  Matrix* 


V»r. 

CSU 

CSC 

CSR 

CSS 

CSl 

CMU 

MSI 

DSC 

NSS 

NST 

NSl 

EFU 

ESU 

ESC 

ESR 

ESS 

EST 

ESI 

RAT 

PAD 

hZ 

1. 

18 

14 

16 

16 

-06 

03 

-04 

08 

11 

13 

-09 

15 

11 

16 

03 

17 

07 

45 

02 

26 

53 

2. 

04 

17 

02 

-02 

10 

15 

05 

02 

11 

25 

34 

10 

-10 

.  16 

20 

09 

12 

47 

12 

06 

61 

3. 

.08 

27 

04 

13 

21 

08 

28 

-22 

17 

28 

07 

12 

08 

50 

24 

30 

07 

26 

-03 

04 

86 

4. 

-10 

48 

24 

23 

10 

18 

03 

01 

09 

20 

14 

05 

03 

38 

06 

18 

25 

16 

03 

17 

76 

5. 

07 

01 

10 

00 

02 

10 

-07 

04 

12 

64 

03 

03 

13 

00 

00 

04 

-01 

-08 

05 

-08 

49 

6. 

21 

00 

23 

26 

19 

-08 

-15 

-01 

24 

03 

-09 

07 

02 

16 

07 

12 

16 

25 

-02 

02 

44 

7. 

-03 

22 

29 

03 

13 

37 

14 

06 

05 

20 

00 

-01 

03 

14 

10 

35 

24 

28 

-01 

06 

65 

8. 

-03 

21 

21 

19 

23 

24 

16 

04 

16 

22 

11 

14 

11 

29 

08 

32 

22 

23 

-08 

03 

69 

9. 

42 

27 

17 

-02 

01 

25 

07 

-14 

u 

13 

06 

01 

29 

33 

05 

10 

18 

04 

05 

03 

64 

10. 

-07 

13 

08 

00 

05 

07 

-03 

03 

18 

28 

03 

03 

05 

39 

05 

22 

40 

25 

03 

10 

59 

U. 

09 

-04 

23 

-08 

02 

-11 

23 

13 

u 

03 

07 

28 

34 

12 

31 

04 

22 

21 

08 

-07 

58 

12. 

53 

06 

-02 

04 

09 

11 

02 

16 

31 

22 

10 

02 

20 

10 

07 

07 

02 

-06 

12 

-06 

57 

13. 

20 

04 

24 

15 

-07 

-12 

00 

-02 

-01 

12 

19 

14 

01 

24 

01 

01 

-02 

18 

-14 

-04 

32 

14. 

13 

08 

31 

-02 

06 

li 

09 

-03 

09 

-25 

49 

20 

10 

16 

20 

03 

06 

10 

-02 

10 

60 

15. 

-04 

13 

08 

04 

10 

06 

01 

09 

18 

15 

15 

62 

26 

•  04 

-10 

10 

05 

14 

02 

•04 

62 

16. 

18 

-09 

23 

25 

08 

15 

23 

18 

16 

13 

09 

11 

26 

17 

21 

27 

45 

08 

-11 

10 

78 

17. 

-03 

05 

17 

23 

17 

22 

-08 

19 

-12 

08 

21 

00 

23 

21 

0,5 

14 

•  18 

35 

01 

06 

55 

16. 

-07 

03 

19 

41 

10 

16 

18 

24 

24 

11 

12 

06 

-03 

.16 

20 

09 

01 

21 

13 

03 

56 

19. 

-04 

-.01 

-13 

11 

17 

-06 

12 

09 

12 

14 

31 

08 

55 

05 

03 

12 

-02 

16 

05 

17 

60 

20. 

-02 

-15 

02 

-10 

05 

-25 

12 

-11 

04 

15 

31 

06 

36 

-02 

-14 

15 

05 

00 

-07 

-02 

43 

21. 

00 

43 

12 

08 

17 

01 

20 

-04 

13 

20 

06 

12 

06 

30 

22 

06 

18 

22 

09 

12 

60 

22. 

-08 

26 

27 

04 

27 

07 

-07 

-07 

18 

24 

10 

09 

15 

26 

16 

32 

28  . 

19 

06 

16 

71 

23. 

-08 

41 

22 

09 

17 

07 

-04 

17 

35 

?4, 

14 

09 

00 

'  41 

04 

25 

03 

19 

-08 

1-2 

,  T9 

24. 

02 

00 

00 

-15 

-01- 

-06 

60 

07 

09 

12 

33 

00 

21 

14 

02 

26 

09 

04 

02 

08 

67 

25. 

-05 

21 

08 

09 

28 

12 

12 

05 

41 

15 

29 

00 

13 

25 

19 

10 

18 

27 

14 

-06 

70 

26. 

-03 

00 

14 

01 

15 

07 

09 

02 

03 

06 

18 

70 

06 

11 

10 

-01 

16 

12 

-03 

04 

65 

27. 

00 

20 

14 

12 

-09 

13 

01 

12 

03 

03 

15 

16 

08 

28 

45 

19 

25 

21 

15 

t13 

63 

28. 

-01 

20 

-05 

-07 

39 

04 

06 

27 

-08 

07 

00 

10 

04 

-05 

-09 

or 

17 

-02 

03 

-09 

35 

29. 

-02 

13 

46 

14 

04 

38 

01 

13 

01 

26 

09 

06 

-03 

08 

07 

24 

13 

14 

00 

08 

60 

30. 

02 

13 

11 

14 

19 

18 

08 

-04 

33 

12 

16 

-05 

-02 

11 

15 

49 

21 

12 

14 

01 

63 

31. 

-13 

17 

03 

-02 

03 

-15 

33 

03 

24 

18 

35 

13 

32 

09 

02 

27 

11 

16 

03 

05 

64 

32. 

-13 

21 

11 

08 

13 

11 

14 

25 

26 

10 

06 

04 

-01 

54 

-02 

15 

08 

13 

02 

-06 

62 

33. 

10 

16 

20 

37 

04 

27 

02 

12 

03 

01 

15 

10 

06 

16 

37 

18 

17 

22 

05 

04 

62 

34. 

28 

21 

07 

02 

05 

39. 

-14 

03 

19 

27 

11 

04 

10 

37 

08 

15 

10 

19 

00 

-18 

68 

35. 

-03 

13 

02 

24 

30 

04 

03 

08 

-02 

06 

14 

21 

01 

29 

22 

37 

16 

23 

03 

-06 

59 

36. 

-02 

21 

07 

-03 

03 

13 

17 

13 

08 

-03 

22 

24 

62 

09 

01 

16 

19 

11 

02 

11 

■71 

37. 

-12 

13 

26 

19 

22 

20 

-05 

-05 

26 

19 

01 

04 

-02 

33 

58 

13 

.05 

13 

-05 

01 

82 

38. 

•  16 

17 

25 

16 

21 

31 

13 

•  10 

25 

08 

-05 

09 

09 

19 

12 

17 

19 

36 

-03 

-22 

69 

39. 

-05 

18 

12 

18 

10 

11 

01 

-06 

18 

10 

29 

-02 

03 

08 

15 

-07 

31 

04 

10 

-03 

58 

40. 

08 

03 

06 

•  12 

08 

-01 

06 

41 

20 

09 

.01 

06 

07 

-05 

-01 

-03 

24 

01 

07 

•  20 

37 

41. 

-17 

23 

18 

13 

04 

16 

34 

•02 

18 

02 

12 

14 

06 

16 

03 

48 

-01 

06 

-05 

-05 

61 

42. 

06 

10 

18 

28 

•  06 

13 

03 

30 

40 

20 

10 

13 

09 

19 

16 

46 

05 

14 

-01 

17 

80 

43. 

00 

03 

29 

-03 

24 

05 

-07 

19 

07 

-03 

06 

03 

04 

33 

27 

19 

21 

23 

-04 

06 

52 

44. 

34 

02 

26 

13 

13 

16 

01 

08 

11 

22 

16 

05 

-01 

25 

-03 

20 

18 

12 

07 

04 

49 

4S. 

12 

06 

32 

05 

36 

•03 

-05 

18 

16 

24 

01 

07 

21 

16 

09 

06 

00 

15 

00 

07 

49 

46. 

05 

20 

35 

21 

•03 

11 

05 

22 

13 

16 

15 

19 

06 

24 

19 

27 

26 

14 

08 

-17 

67 

47. 

37 

09 

16 

04 

14 

14 

15 

03 

24 

43 

-02 

06 

07 

26 

10 

14 

13 

08 

07 

08 

62 

48. 

06 

05 

22 

09 

17 

01 

-04 

04 

02 

21 

-02 

-05 

12 

12 

04 

15 

-10 

07 

50 

.06 

46 

49. 

03’ 

-15 

22 

07 

-09 

-02 

03 

-07 

-05 

-04 

-13 

01 

09 

08 

01 

-01 

-13 

-09 

55 

07 

46 

51. 

-09 

04 

-03 

01 

-04 

14 

05 

-15 

04 

-13 

07 

00 

01 

•  02 

-01 

00 

18 

05 

-03 

56 

43 

52. 

06 

24 

17 

11 

13 

61 

-08 

-02 

•01 

14 

-06 

07 

11 

30 

02 

26 

17 

28 

-01 

02 

81 

53. 

05 

-11 

26 

15 

-06 

63 

-09 

-20 

19 

38 

-09 

14 

18 

20 

06 

19 

23 

03 

-13 

18 

98 

54. 

20 

15 

15 

-12 

05 

70 

03 

-08 

25 

10 

06 

07 

•  16 

19 

24 

13 

10 

07 

-12 

11 

86 

‘Decimal  polnte  omitted. 


RESULTS  OF  THE  FACTOR  ANALYSIS 


be  avoided.  The  result  of  the  fifth  target-oriented  ro¬ 
tation  upon  the  (iTincipal-axes  matrix  was  accepted  as 
the  final  solution  and  is  presented  in  Table  5  as  the 
rotated  factor  matrix. 


After  the  final  rotation,  differences  between  the 
unrotated  and  rotated  communalities  were  no  larger 
than  ,  02;  most  of  the  differences  were  .  00.  Since  fur¬ 
ther  extensive  checking  seemed  unnecessary,  the  reduced 
correlation  matrix  was  not  produced  by  post-multiplying 
the  rotated  factor  matrix  by  its  transpose.  A  check 
involving  five  randomly  selected  correlation  coefficients 
showed  that  the  reduced  coefficients  are  within  .  02  of 
their  respective  empirical  coefficients. 


The  interpretation  of  each  of  the  20  rotated  factors 
is  based  upon  the  hypothesized  factor  content  of  the  tests 
loading  .  30  or  more  upon  the  factor.  In  some  interpre¬ 
tations  to  follow,  the  factors  are  defined  by  tests  not 
hypothesized  to  cohere.  In  these  cases,  the  factor  in¬ 
terpretations  are  dependent  not  only  upon  the  hypothe¬ 
sized  factor  contents  of  the  tests,  but  also  upon  their 
newly  revealed  common  properties. 


Each  factor  in  Table  5  is  discussed  in  the  order 
in  which  it  was  discussed  in  the  section  on  the  hypothesis, 
The  test  loadings  for  each  successive  factor  will  be 
listed  along  with  all  additional  loadings  (,  30  or  higher) 
of  the  tests,  if  they  proved  to  be  complex. 
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Interpretation  of  the  Reference  Factors 


CSU  Cognition  of  symbolic  units 

12.  Disemvowelled  Words  (CSU) 

.  53 

(.  31  NSS) 

9.  Correct  Spelling  (ESU) 

.  42 

(.  33  ESC) 

47.  Word  Transformation  (NST) 

.  37 

(.  43  NST) 

44.  Word  Combinations  (CSU) 

.  34 

clearly  defined,  composite,  classes  factor. 
CSR  Cognition  of  symbolic  relations 


29.  Seeing  Trends  II  (CSR)  . 

.46 

(.  38  CMU) 

46.  Word  Relations  (CSR) 

.  35 

45.  Word  Patterns  (CSI) 

.  32 

(.  36  CSI) 

-14.  Form  Reasoning  (NSI) 

.  31 

(.  49  NSI) 

Disemvowelled  Words,  although  shortened,  once 
again  leads  the  tests  that  are  loaded  on  the  CSU  factor. 
It  would  appear  from  the  test's  history  that  the  factor 
is  concerned  with  the  recognition  of  complete  and  correct 
words;  a  factor  that  might  be  called  "word  closure" 
(Pemberton,  1953). 

Such  a  conclusion  is  strengthened  by  the  fact  that 
Correct  Spelling,  designed  as  a  measure- of  ESU,  is  also 
a  measure  of  the  recognition  (perhaps  not  in  a  critical 
light)  of  complete  and  correctly  spelled  words.  Correct 
Spelling  was  hypothesized  to  be  a  test  of  ESU  because 
sensitivity  to  the  correctness  or  incorrectness  of  com¬ 
monly  misspelled  words  was  thought  to  be  evaluative. 
Possible  reasons  for  its  being  a  CSU  teat  instead  will 
be  offered  in  the  discussion  of  factor  ESU. 

Word  Transformation's  loading  on  the  CSU  factor 
implies  that  the  careful  analysis  of  previous  test  data 
did  not  accomplish  all  that  it  intendeds  Tests  for  both 
CSU  and  NST  were  correlated  in  the  previous  analysis 
(Guilford,  et  al.  ,  1961)  in  which  those  factors  were  dis¬ 
covered.  From  Word  Transformation's  correlation 
with  the  CSU  factor,  although  secondary,  it  appears 
that  the  symbolic  redefinition  task  is  dependent  upon 
recognition  of  the  symbolic  units  needed  in  effecting 
the  transformation.  Word  Combinations  once  again 
has  its  primary  loading  on  CSU  with  no  significant  load¬ 
ings  on  the  NST  factor  for  which  it  had  been  originally 
designed. 

CSC  Cognition  of  symbolic  classes 

4.  Best  Number  Pairs  (ESC)  .48  (.  38  ESC) 

21.  Number  Classification  (CSC)  .43  (.  30  ESC) 

23.  Number.Oroup  Naming  (CSC)  .41  (.  41  ESC;  .  38  NSS) 

The  two  shortened  forms  of  the  tests  selected  to 
measure  the  CSC  factor  function  as  anticipated,  except 
that  the  tests  for  CSC  are  led  by  a  test  designed  for  ESC, 
Like  its  analogous  semantic  test.  Best  Word  Pairs 
(Nihira,  et  al.  ,  1964),  Best  Number  Pairs  contributes 
more  to  the  cognition-of-classes  factor  than  to  the  eval¬ 
uation  factor.  Like  Best  Word  Pairs.  Best  Number 
Pairs  asks  E  to  evaluate  which  pair  of  stimuli  makes 
the  best  class,  but  the  specific  properties  of  the  best 
class,  in  other  words,  the  specific  criteria,  are  not 
defined  in  each  item.  Since  the  specific  aspects  of  the 
best  class  need  to  be  discovered  for  each  item,  cog¬ 
nition  abilities  should  be  expected  to  determine  much 
of  the  test's  variance.  The  CSC  factor  defined  in  this 
analysis  could  be  confined  to  the  ability  to  recognize 
common  properties  of  numbers,  but  a  letter  test  has 
previously  been  loaded  on  it  (Guilford,  et  al.  ,  1961). 

The  significant  side  loadings  of  the  three  CSC 
tests  on  the  ESC  factor  impressively  demonstrate  that 
testa  of  the  two  factors  are  highly  related.  Although 
it  could  be  contended  that  no  separation  between  CSC 
and  ESC  tests  should  have  been  attempted  in  rotating 
axes,  it  was  felt  that  an  unclear  separation  would  be 
heuristically  more  valuable  than  the  report  of  a  less 


Both  CSR  tests  function  as  anticipated  in  this  anal¬ 
ysis.  Although  they  had  undergone  considerable  short¬ 
ening,  it  appears  that  their  variances  remain  dominated 
by  CSR.  The  significant  CMU  side  loading  for  Seeing 
Trends  II  is  not  reasonable  as  the  trends  are  based 
solely  on  the  letter  content  of  the  words,  and  not  on 
their  meanings.  The  result  is  consistent  with  a  similar 
CMU  side  loading  found  for  Seeing  Trends  II  by  Guilford, 
et  al.  (1961).  Some  semantic  recognition  must  somehow 
be  involved  in  the  test.  Although  recognition  of  the 
symbolic  trends  is  indicated  by  verbal  statement  of  E, 
the  scoring  procedure,  did  not  weight,  but  in  a  minimum 
manner,  E's  verbal  abilities  at  expressing  the  symbolic 
trend. 

Word  Patterns  and  Form  Reasoning,  measures 
of  CSI  and  NSI  respectively,  have  small  but  significant 
side  loadings  on  CSR.  This  is  not  the  first  time  that  a 
separation  between  relations  tests  and  implications 
tests  has  been  difficult  to  achieve  (see  Gershon,  et  al.  , 
1963).  Each  of  these  tests  did  have  its  primary  loading 
on  the  factor  for  which  it  was  hypothesized. 

CSS  Cognition  of  symbolic  systems 

18.  Letter  Triangle  (CSS)  .  41 

33.  Similar  Pairs  (ESR)  .  37  (.  37  ESR) 

6.  Circle  Reasoning  (CSS)  .  26 

Letter  Triangle  leads  the  factor  called  CSS  in  this 
analysis.  Although  both  Letter  Triangle  and  Circle 
Reasoning  defined  CSS  factors  with  moderate  loadings 
in  several  previous  investigations  (Green,  et  al.  ,  1953; 

Guilford,  et  al.  ,  1954;  Kettner,  et  al.  ,  1959:  Guilford, 
et  al.  ,  1961),  they  did  not  emerge  together  in  a  more 
recent  investigation  by  Petersen,  et  al.  (1963).  The 
implication  derived  from  the  unsuccessful  attempts  to 
isolate  a  clear  CSS  doublet  is  that  additional  hypothe¬ 
sized  measures  of  CSS  should  be  developed  or  refined 
and  analyzed  in  the  future  so  that  several  measures 
would  be  available  that  consistently  cohere. 

Similar  Pairs'  loading  on  the  CSS  factor  should 
be  accounted  for  by  cognition  that  might  be  necessary 
to  recognize  the  relations  within  word  pairs.  The  de¬ 
termination  of  these  relations  is  dependent  upon  alpha¬ 
betical  order,  which  also  determines  the  systems  in 
Letter  Triangle,  and  upon  need  for  letter  rearrangements 
in  some  items. 

CSI  Cognition  q{  symbolic  implications 

28.  S  Test  (ESI)  .  39 

45.  Word  Patterns  (CSI)  .  36  (.  32  CSR) 

35.  Symbol  Grouping  (CSI)  .  30  (.  37  ESS) 

The  ability  to  foresee  symbolic  implications,  CSI, 
is  defined  in  this  analysis  by  the  two  tests  designed  to 
measure  the  factor,  but  is  measured  best  by  the  S  Test, 
a  test  originally  designed  to  measure  sensitivity  to  prob¬ 
lems,  The  original  F  Test,  from  which  the  S  Test  was 
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derived,  probably  did  not  indicate  the  "sensitivity-to- 
problems"  factor  because  the  factor  was  semantic.  The 
semantic  tests  that  did  bring  out  the  sensitivity-to-prob- 
lems  factor,  were  later  shown  to  belong  largely  on  factor 
CMI,  the  cognition  ability  parallel  to  CSI,  on  which  we 
now  see  the  S  Test  loaded.  It  appears  that  there  is  no 
difference  between  the  ability  to  see  symbolic  implica¬ 
tions  and  being  sensitive  to  them. 

The  two  CSI  tests.  Word  Patterns  and  Symbol 
Grouping,  althovigh  shortened  to  make  them  more  univo¬ 
cal,  still  appear  to  be  complex.  Symbol  Grouping's 
primary  loading  on  ESS  appears  to  reflect  the  systems 
(ordering)  aspect  of  the  items;  some  items  require 
relatively  little  foresight  and  more  systematic,  almost 
rote,  grouping  of  symbols.  Evaluation  probably  enters 
because,  in  attempting  to  achieve  the  most  efficient 
solution,  E  needs  to  compare  one  solution  with  another 
and  decide  which  is  more  efficient. 

CMU  Cognition  of  semantic  units 


54. 

SCAT  Verbal  (CMU) 

70 

53. 

PSAT  Verbal  (CMU) 

63 

(.  38  NST) 

52. 

ITED  General  Vocabulary  CMU) 

61 

(.  30  ESC) 

34. 

Sound  Grouping  (ESC) 

39 

(.  37  ESC) 

29. 

Seeing  Trends  II  (CSR) 

38 

(.  46  CSR) 

7. 

Correct  Letter  Orders  (ESS) 

,  37 

(.  35  ESS) 

38. 

Symbol  Reasoning  (ESI) 

.  31 

(.  36  ESI) 

The  verbal-comprehension  factor  is  clearly  defined 
by  the  three  tests  selected  to  measure  CMU.  The  tests 
are  relatively  univocal,  having  very  high  loadings  on 
CMU  and  small  or  no  loadings  on  symbolic  factors.  The 
tendency  of  the  CMU  tests  to  be  loaded  about.  20  on  many 
of  the  remaining  factors  in  the  analysis  probably  is  ac¬ 
counted  for  by  variance  introduced  in  the  understanding 
of  instructions  or  in  verbalizing  aspects  of  solving  sym¬ 
bolic  problems. 

Once  again,  Sound  Grouping  demonstrates  its  fac¬ 
torial  complexity  by  splitting  its  variance  between  ESC 
and  CMU.  Familiarity  with  the  words  used  as  stimuli 
lor  this  test  appears  to  facilitate  either  the  pronunci¬ 
ation  or  the  classification  based  upon  the  pronunciations. 
One  might  expect  an  analogous  phenomenon  underlying 
the  CMU  loading  of  Seeing  Trends  IL  But  knowledge  of 
meanings  of  the  words  in  Seeing  Trends  II  in  no  way  aids 
in  the  discovery  of  the  symbolic  trend.  The  only  com¬ 
mon  element  in  the  CMU  tests  and  Seeing  Trends  II  is 
that  words  are  employed. 

Correct  Letter  Order's  CMU  loading  is  rational ly 
tenuous,  the  only  CMU  variance  possibly  being  contri¬ 
buted  by  the  verbal  descriptions  of  the  letter  series. 
Symbol  Reasoning's  almost  negligible  CMU  loading  ap¬ 
pears  to  be  fortuitous,  as  no  items  employ  words  or 
verbally  complex  criteria.  It  seems  possible,  however, 
that  Es  may  translate  the  symbolic  syllogisms  into  some 
verbal  form  and  then  perform  the  required  task. 

MSI  Memory  for  symbolic  implications 

24.  Numerical  Operations  (MSI)  .60  (.  33  NSI) 

41.  Way-Out  Numbers  (ESS)  .34  (.48  ESS) 

31.  Sign  Changes  (NSI)  .  33  (.  35  NSI;  .  32  ESU) 

Numerical  Operations  emerges  clearly  as  the  test 
defining  the  MSI  factor.  MSI  receives  further  support 


from  Way-Out  Numbers  due  to  the  errors  initemcon- 
struction  mentioned  earlier,  and  from  Sign  Changes,  a 
test  designed  to  measure  NSI,  but  whichhas  consistently 
shared  MSI  variance  (Petersen,  et  al.  ,  1963).  It  ap¬ 
pears  that  the  MSI-NSI  sharing  is  reciprocal;  while 
Sign  Changes  is  loaded  slightly  on  MSI,  Numerical  Op¬ 
erations  is  loaded  slightly  on  NSL  The  shared  variance 
on  these  two  factors  is  not  great  enough,  however,  to 
cause  serious  concern  regarding  their  distinctness. 

DSC  Divergent  production  of  symbolic  classes 

40.  Varied  Symbols  (DSC)  .41 

42.  Word  Changes  (NSS)  .30  (.  40  NSS;  .  46  ESS) 

Varied  Symbols  once  again  serves  as  the  principal 
measure  of  the  DSC  factor.  Number  Grouping,  which 
was  not  loaded  on  DSC  in  a  previous  investigation 
(Gershon,  et  al.  ,  1963),  failed  to  be  again.  The 

strong  lace  validity  of  Number  Grouping  as  a  measure 
of  DSC  apparently  is  misleading  as  the  test  loaded  on 
the  ESS  factor  in  this  analysis.  No  explanation  could 
be  found  for  the  small  tertiary  DSC  loading  of  Word 
Changes  other  than  that  Es  may  employ  a  discursive 
trial-and-error  ordering  procedure  which  would  mean 
divergent-thinking  processes. 

NSS  Convergent  production  of  symbolic  systems 

25.  Operations  Sequence  (NSS)  .41 

42.  Word  Changes  (NSS)  .  40  (.  46  ESS;  .  30  DSC) 

23.  Number-Croup  Naming  (CSC)  .35  (.41  CSC;  .41  ESC) 
30.  Series  Relations  (ESS)  .33  (.49  ESS) 

12.  Disemvowelled  Words  (CSU)  .31  (.  53  CSU) 

The  two  tests  selected  to  measure  NSS  perform 
again  in  a  reliable  manner.  It  appears  that  Operations 
Sequence  is  one  of  the  strongest  and  is  the  most  univocal 
measure  of  this  factor.  The  large  side  loading  of  Word 
Changes  on  ESS  might  be  due  to  a  strategy  that  employs 
alternate  orderings  that  are  quickly  evaluated  according 
to  the  limitations  imposed  by  criteria  given  in  the  test 
instructions.  A  similar  rationale  could  explain  the 
minor  secondary  NSS  loading  of  Series  Relations.  E 
tries  out  each  given  operation  in  turn,  producing  a  fully 
determined  series. 


The  small  but  significant  loadings  of  Number- 
Group  Naming  and  Disemvowelled  Words  on  NSS  cannot 
be  explained,  as  neither  test  appears  to  involve  system 
production. 

NST  Convergent  production  of  symbolic  transformations 

5.  Camouflaged  Words  (NST)  .  64 
47.  Word  Transformation  (NST)  .  43  (.  37  CSU) 

53.  PSAT  Verbal  (CMU)  .  38  (.  63  CMU) 

The  NST  factor  emerged  in  this  analysis  with  both 
tests  selected  to  measure  it  loading  primarily  upon  it. 
The  CSU  side  loading  of  Word  Transformations  was 
discussed  in  connection  with  the  CSU  factor.  From 
the  PSAT  Verbal  test's  loading  on  NST,  it  appears  that 
NST  may  play  a  role  in  responding  to  verbal  tests  — 
the  emergence  of  words  or  word  roots  may  aid  in  the 
comprehension  of  word  meanings  and  complex  ideas. 
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NSI  Convergent  production  of  symbolic  implications 


14.  Form  Reasoning  (NSI)  .49  (.31CSR) 

31.  Sign  Changes  (NSI)  .  35  (.  33  MSli  .  32  ESU) 

2.  Best  Letter  Set  (ESS)  .34  (.47  ESI) 

24.  Numerical  Operations  (MSI)  .  33  (.  60  MSI) 

20.  Marking  Speed  (MOT)  .31  (.  36  ESU) 

19.  Letter  "U"  (ESU)  .  31  (.  55  ESU) 


The  two  tests  selected  to  measure  NSI  perform 
in  this  analysis  as  expected,  except  that  both  have  sane 
complexity.  In  addition  to  the  NSI  tests'  being  complex 
the  NSI  factor  found  in  this  analysis  shows  minor  load¬ 
ings  in  a  few  heterogeneous  types  of  tests.  The  factor 
is  defined  in  part  by  an  MSI  test  that  has  previously 
shared  NSI  variance  (Petersen,  et  al.  ,  1963),  and  had 
been  actually  thouglit  to  be  an  NSI  measure  for  a  long 
time.  Shifting  it  primarily  to  the  MSI  factor  resulted 
from  finding  that  it  is  strongly  related  to  the  Wechsler 
Digit  Symbol  test,  which  is  more  obviously  a  memory 
test  (Davis,  1956;  deMille,  1962).  In  addition,  Marlcing 
Speed  and  Letter  "U",  both  of  which  are  speeded  tests, 
share  variance  with  NSI,  apparently  due  to  their  com¬ 
mon  speeded  natures.  Had  the  rotational  solution  in¬ 
sisted  upon  a  separate  speed  factor,  with  Marking  Speed 
the  leading  variable  on  it,  this  test  would  probably  not 
have  appeared  in  this  list,  or  in  the  list  for  factor  ESU 
to  be  presented  a  bit  later.  Best  Letter  Set's  loading 
on  NSI  cannot  be  logically  explained  by  common  aspects 
of  the  tests;-  there  appear  to  be  none. 


should  go  with  Thurstone's  original  perceptual  factor  • 
or  should  represent  a  separate  factor  (Thur stone,  1938b; 
Coombs,  1941;  Bechtoldt,  1947). 

The  third  ESU  test  loaded  on  the  ESU  factor  is 
Derivations,  also  a  sensitivity  test,  involving  words. 
Based  upon  the  three  ESU  tests  loading  on  tliis  factor, 
it  appears  that  ESU  is  the  ability  to  make  rapid  deci¬ 
sions  regarding  the  symbolic  identity  or  accuracy  of 
words,  letter  sets,  and  number  sets.  In  Symbol  Iden- 
titie s ,  there  is  a  comparison  of  two  given  symbolic 
units  to  determine  whether  or  not  they  are  identical. 
In  Letter  "U",  a  word  class  is  specified  (words  con¬ 
taining  the  letter  "U").  with  E  to  say  whether  or  not 
each  word  satisfies  the  specification.  Symbol  Identi¬ 
ties  is  a  direct  parallel  to  figural  tests  of  EFU,  in  which 
figures  are  to  be  compared,  with  E  to  decide  whether 
or  not  they  contain  exactly  the  same  elements.  Letter 
"U"  is  parallel  in  some  important  respects  with  Double 
Descriptions  (Nihira,  et  al.  ,  1964).  In  that  test,  E  is 
given  a  class  specification  for  a  verbalized  concept, 
e,  g.  ,  things  both  hard  and  round,  and  he  must  say  which 
of  four  offered  semantic  units  best  fulfills  the  specifi¬ 
cations.  Besides  the  difference  in  kind  of  information, 
which  determines  two  distinct  abilities,  there  is  another 
difference,  which  probably  is  not  very  important,  and 
that  is  the  yes-no  type  of  decision  (sensitivity)  in  Letter 
" U"  and  the  relative  decision  (estimation)  in  Double 
Description. 


To  date,  this  solution  represents  the  clearest 
separation  between  the  MSI  and  NSI  factors,  although 
there  still  seems  to  be  some  common  aspect  in  the  tests 
used  here  to  measure  them  and  more  univocal  tests  are 
apparently  needed  for  both. 

EFU  Evaluation  of  figural  units 

26.  Perceptual  Speed  (EFU)  .  70 

15.  Identical  Forms  (EFU)  .62 

The  two  EFU  tests  perform  exactly  as  hypothe¬ 
sized,  being  univocal  and  highly  saturated  with  common- 
factor  variance.  The  EFU  factor  is  the  clearest  inter¬ 
pretable  factor  to  emerge  in  this  analysis,  probably  due 
to  its  relative  dissimilarity  of  content  from  the  symbolic 
and  semantic  factors. 


Interpretation  of  the  Experimental  Factors 
ESU  Evaluation  of  symbolic  units 


36.  Symbol  Identities  (ESU) 

19.  Letter  "U"  (ESU) 

20.  Marking  Speed  (MOT) 
11.  Derivations  (ESU) 

31.  Sign  Changes  (NSI) 


.  62 

.  55  (.  31  NSI) 

.  36  (.  31  NSI) 

.  34  (.  31  ESR) 

.  32  (.  35  NSI;  .  33  MSI  ) 


The  ESU  factor  emerged  with  remarkable  clarity. 
The  two  speeded  sensitivity  tests  that  were  previously 
suggested  as  measures  of  ESU  (Guilford  and  Hoepfner, 
1963)  lead  the  factor  with  high  loadings  and  little  or  no 
complexity.  It  is  interesting  to  note  that  with  stropg 
tests  of  EFU  and  a  sufficient  number  of  tests  for  ESU, 
including  Symbol  Identities  and  Letter  "U",  in  the  anal¬ 
ysis,  the  two  factors  separate  very  clearly.  This  de¬ 
cisive  result  clears  up  earlier  uncertainties  as  to  wheth¬ 
er  ''perceptual-speed"  tests  composed  of  literal  material 


Derivations  does  not  fit  either  of  the  two  item 
models  just  described  for  Symbol  Identities  and  Letter 
"U".  The  things  being  compared  are  not  exactly  the 
same  except  for  one  element,  as  in  the  former,  nor  are 
class  specifications  given,  as  in  the  latter.  The  letters 
of  the  short  word  said  to  be  extracted  from  the  long  word 
must  coincide  with  a  completely  identical  set  of  letters 
in  the  long  word,  except  that  the  order  is  probably  dif¬ 
ferent.  There  is  no  clear  model  presented  for  com¬ 
parison.  This  may  be  a  reason  for  the  lower  ESU  load¬ 
ing  for  Derivations  than  for  Symbol  Identities. 

The  two  ESU  "misses,  "  those  tests  hypothesized 
for  ESU  but  not  loaded  significantly  on  it  (Correct  Spell¬ 
ing  and  Familiar  Letter  Combinations),  also  aid  in  in¬ 
terpreting  the  ESU  factor  by  indicating  what  ESU  is  not. 
One  characteristic  the  two' "misses"  have  in  common 
but  do  not  share  with  the  other  three  ESU  tests,  is  that 
the  things  with  which  comparisons  must  be  made  are 
not  given  on  the  printed  page.  They  can  only  be  com¬ 
pared  with  something  in  memory  storage  or  something 
retrieved  from  memory  storage,  perhaps  in  the  form 
of  an  image.  In  Correct  Spelling,  the  needed  model  is 
the  remembered  correct  spelling  of  each  word.  Some 
of  these  models  E  would  have  and  some  he  would  not. 
The  task  boils  down  to  the  question  of  how  many  of  the 
120  words  in  the  test  does  E  know,  spelled  correctly. 
This  statement  of  the  task  makes  it  appear  like  a  mea¬ 
sure  of  cognition,  as  it  turned  out  by  analysis  to  be. 

In  Familiar  Letter  Combinations,  the  criterion 
for  judgment  is  familiarity  of  the  syllables  or  their 
observed  probability  of  occurrence  in  E's  experience. 
In  this  test,  there  is  no  clear  model  for  E  to  use,  and 
wliat  he  has  to  use  is  also  something  in  or  from  memory 
storage.  Although  it  appears  that  the  memory  feature 
applies  especially  to  the  two  £1SU  tests  that  missed,  the 
question  arises  as  to  how  general  the  implied  evalua¬ 
tion  principle  is.  If  it  is  quite  general,  the  principles 
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that  comparisons  must  be  between  perceived  information 
would  place  an  important  restriction  upon  the  definition 
of  evaluation  abilities. 

The  presence  of  Marking  Speed,  along  with  Sign 
Changes,  here  suggests  some  confounding  of  a  finger- 
speed  factor  with  both  NSI  and  ESU.  As  was  suggested 
in  the  discussion  of  NSI,  the  rotation  to  an  additional 
finger- speed  factor  might  have  cleared  up  the  picture 
for  both  NSI  and  ESU.  Implied  is  a  general  principle, 
namely,  that  the  appearance  of  obliqueness  among  fac¬ 
tors  may  be  due  to  lack  of  a  sufficient  number  of  di¬ 
mensions  being  included  in  orthogonal  rotations. 

ESC  Evaluation  of  symbolic  classes 


32. 

3. 

Sign  Changes  II  (ESR) 

Best  Number  Class  (ESC) 

54 

50 

(.  30  ESS) 

23. 

Number -Group  Naming  (CSC) 

41 

(.  41  CSC;  .  35  NSS) 

10. 

Decoding  (EST) 

39 

(.  40  EST) 

4. 

Best  Number  Pairs  (ESC) 

38 

(.  48  CSC) 

34. 

Sound  Grouping  (ESC) 

37 

(.  39  CMU) 

43. 

Word  Choice  (ESC) 

33 

37. 

Symbol  Manipulation  (ESR) 

33 

(.  58  ESR) 

9. 

Correct  Spelling  (ESU) 

33 

(.  42  eSU) 

21. 

Number  Classification  (CSC) 

30 

(.  43  CSC) 

52. 

ITED  General  Vocabulary  PMU) 

30 

(.  61  CMU) 

The  test.  Decoding,  designed  as  a  measure  of  EST, 
also  is  loaded  equally  on  the  ESC  factor.  The  reason 
for  its  loading  is  probably  due  to  the  class-properties 
variance  that  is  introduced  in  E's  analysis  of  the  words 
into  classes  according  to  the  rules  for  the  codes,  each 
rule  representing  a  different  class  property.  It  is 
necessary  to  recognize  the  parts  of  the  uncoded  words 
as  belonging  to  certain  classes  of  codability  and  then  to 
make  decisions  as  to  the  values  of  the  recognized  classea 

The  three  ESC  tests  with  the  smaller  significant 
loadings  on  this  factor  were  constructed  as  estimation 
tests  of  ESC.  These  tests,  except  for  Best  Number 
Pairs,  involve  making  a  choice  among  given  class  pro¬ 
perties  on  the  basis  of  criteria  supplied  in  the  test.  Al¬ 
though  it  would  seem  reasonable  to  have  rotated  the  ESC 
axis  to  maximize  its  correlations  with  these  more  sim¬ 
ple  ESC  tests,  their  tendency  toward  factor  complexity 
and  theresulting  loss  of  simple  structure  weighed  against 
such  a  move. 

The  loadings  of  Symbol  Manipulation,  Correct 
Spelling,  and  ITED  General  Vocabulary  on  the  ESC  fac - 
tor  are  small  and  not  important  to  the  iiiterpretation  of 
this  factor  when  the  distinctly  larger  loadings  on  their 
respective  hypothesized  factors  are  considered. 


Although  all  four  tests  designed  to  be  measures 
of  the  ESC  factor  are  loaded  on  the  factor  called  ESC, 
the  factor  is  the  least  clear  of  all  the  new  experimental 
factors  found.  Three  of  the  ESC  tests  have  complexi¬ 
ties  of  two,  and  the  factor  is  led  by  a  test  designed  for 
ESR.  Usually,  inspection  of  the  common  characteris¬ 
tics  of  the  tests  leading  a  factor  reveals  the  essential 
nature  of  the  common  aspect  that  the  factor  represents. 
In  the  case  of  the  ESC  factor,  the  common  element  of 
the  leading  tests  is  rather  obscure. 

Sign  Changes  II  requires  the  examinee  to  change 
signs  in  a  numerical  expression  so  that  the  expression 
becomes  an  equation.  Introspectively,  it  seems  that  a 
successful  attack  on  such  problems  would  include  the 
tactic  of  becoming  aware  of  what  both  sides  of  the  ex¬ 
pression  have  potentially  in  common,  and  from  this 
awareness,  to  make  the  appropriate  sign  changes  to 
bring  about  that  common  numerical  value  and  thus  change 
the  expression  into  an  equation.  To  clarify  this  attack 
with  an  example,  consider  the  sample  item:  3-11  =  6x2. 
The  first  step  in  effectively  changing  this  expression 
into  an  equation  is  not  to  substitute  signs,  but  to  be  sen¬ 
sitive  to  what  the  pair  of  numerical  value  3  and  1,  and 
the  pair  6  and  2  potentially  have  in  common.  Their 
common,  or  class  property  is  either  the  numerical  value 
of  4  (3  -1  1  and  6  -  2),  or  3  (3  x  1  and  6  -i-  2).  Since  only 
one  of  the  necessary  sign  changes  is  given  as  an  alter¬ 
native  answer,  the  only  acceptable  solution  is  the  first 
one,  and  the  comrnon  element  in  the  expression  is  the 
value  of  4,  the  value  for  which  the  signs  must  be  changed. 
But  this  line  of  thinking  suggests  cognition  rather  than 
evaluation,  factor  CSC  rather  than  ESC,  and  the  loading 
for  this  test  on  CSC  is  only  .  17.  Another  hypothesis  is 
that  E  somehow  takes  the  offered  solutions  as  classes 
of  operation  changes  and  considers  them  for  adequacy. 

The  task  involved  in  Best  Number  Class  is  more 
definitely  to  realize  the  numerical  classifications  of 
given  numbers  and  then  to  select  the  one  classification 
that  is  most  valuable,  value  of  number  classes  being 
defined  by  the  test  as  the  criterion. 


ESR  Evaluation  of  symbolic  relations 


37.  Symbol  Manipulation  (ESR) 

.  58 

(.  33  ESC) 

27.  Related  Worde  I  (ESR) 

.  45 

33.  Similar  Pairs  (ESR) 

.  37 

(.  37  CSS) 

11.  Derivations  (ESU) 

.  31 

(.  34  ESU) 

Three  of  the  four  tests  designed  to  measure  ESR 
are  loaded  on  the  ESR  factor,  clearly  defining  it  as  rep¬ 
resenting  the  ability  to  make  choices  among  symbolic 
relationships  on  the  basis  of  identity  and  consistency. 
The  significant  side  loadings  of  the  ESR  tests  were  men¬ 
tioned  before  in  connection  with  the  respective  factors 

upon  which  the  complex  ESR  tests  are  loaded - factors 

ESC  and  CSS. 

Derivations'  small  loading  on  ESR  could  be  due  to 
the  whole-part  relations  between  the  units  in  the  test. 
Short  words  were  to  be  judged  as  being  parts  of  longer 
words  or  not  being  parts.  When  small  or  simple  words 
are  to  be  judged  as  derived  or  not  derived  from  the 
given  word,  a  choice  based  upon  such  a  part-whole  re¬ 
lationship  is  probably  more  efficient  than  a  comparison 
of  the  elements  in  the  words  for  unit  identities, 

ESS  Evaluation  of  symbolic  systems 


30.  Series  Relations  (ESS) 

41.  Way-Out  Numbers  (ESS) 

42.  Word  Changes  (NSS) 

35.  Symbol  Grouping-  (CSI) 

7.  Correct  Letter  Orders  (ESS) 

8.  Correct  Number  Series  (ESS) 
22.  Number  Grouping  (DSC) 


.  49  (.  33  NSS) 

.  48  (.  34  MSI) 

.46  (.40  NSS;  .30  DSC) 
.  37  (.  30  CSI) 

.  35  (.  37  CMU) 

.  32 
.  32 


Four  of  the  five  tests  designed  for  ESS  came  out 
significantly  loaded  on  ESS.  The  two  leading  tests. 
Series  Relations  and  Way-Out  Numbers,  are  in  the  esti¬ 
mation  category  and  are  composed  of  numbers.  The 
two  with  distinctly  smaller  loadings.  Correct  Letter 
Orders  and  Correct  Number  Series,  are  in  the  sensi¬ 
tivity  category,  one  being  a  number  test  and  the  other 
a  letter  test.  It  may  be  of  some  interest  that  while 
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sensitivity  tests  proved  to  be  better  for  ESU,  esti¬ 
mation  tests  proved  better  for  ESS,  The  trend  must 
be  better  supported,  however,  before  a  principle  can 
be  stated.  It  is  probably  significant,  however,  that  a 
system  can  deviate  from  a  standard  of  comparison  much 
more  readily  by  degrees  than  can  a  unit. 

Series  Relations  and  Way-Out  Numbers  differ 
somewhat  in  the  operations  that  they  require.  In  the 
former,  E  probably  makes  new  systems  (series),  fol¬ 
lowing  rules  given  in  the  alternative  responses.  He 
then  compares  each  new  system  with  the  model  that  is 
given,  deciding  which  new  one  comes  nearest.  The 
criterion  is  the  degree  of  closeness  of  one  set  of  num¬ 
bers  to  another  set.  In  the  latter  test,  he  is  virtually 
to  compare  the  distances  of  the  first  and  last  numbers 
in  an  irregular  series  to  decide  which  one  is  farther 
from  the  other  numbers.  It  is  as  if  he  were  treating 
the  same  series  first  as  one  system  and  then  as  another, 
or  from  one  point  of  view  then  from  another.  The  cri¬ 
terion  is  numerical  distance.  The  difference  between 
these  two  relatively  strong  ESS  tests  contributes  some 
breadth  of  the  nature  of  the  factor. 

The  two  weaker  ESS  tests  are  close  to  being  al¬ 
ternate  forms  of  the  same  test,  and  both  differ  from  the 
two  stronger  tests,  as  indicated  before.  They  present 
letter  series  in  the  one  case  and  number  series  in  the 
other,  with  a  verbal  description  of  the  principle  that 
should  be  satisfied  in  a  series.  Sometimes  the  series 
follows  the  principle  exactly,  sometimes  not.  The 
sample  for  a  letter  series  given  in  the  Appendix  de¬ 
scribes  the  series  thus:  "Alternate  letters  in  the  alpha¬ 
bet  (skipping  one),  "  The  sample  for  a  number  series 
is:  "Alternately  add  1,  multiply  by  3."  It  is  probably 
significant  that  Correct  Letter  Orders  has  a  significant 
loading  on  CMU  whereas  Correct  Number  Series  does 
not.  The  illustrated  rules  are  somewhat,  although  not 
universally,  typical.  But  it  should  matter  more  whether 
verbal  terms  are  correctly  understood  in  the  first  exam¬ 
ple  than  in  the  second.  In  the  latter  the  rule  can  be  more 
simply  and  precisely  stated. 

Word  Changes'  primary  loading  on  ESS  was  ex¬ 
plained  earlier  as  possibly  resulting  from  a  strategy  of 
rapidly  ordering  the  given  words  and  then  evaluating  the 
orders  to  determine  whether  they  arc  consistent  with 
the  rules  of  the  test.  In  this  analysis,  the  evaluation 
variance  proved  to  be  fully  as  strong  as  the  convergent- 
production  variance.  In  a  similar  manner,  Symbol 
Grouping' s  ESS  loading  was  explained  as  possibly  re¬ 
sulting  from  evaluating  trial  orderings  of  the  symbols 
to  determine  whether  they  are  consistent  with  the  rules 
and  are  most  efficient  for  achieving  the  final,  accepted 
ordering.  This  kind  of  result  suggests  the  generaliza¬ 
tion  that  a  number  of  cognitive  and  convergent-production 
tests  may  actually  involve  some  evaluation  variance 
which  has  heretofore  been  missed  because  they  have  not 
been  analyzed  along  with  definitive  evaluation  tests. 

Number  Grouping's  only  significant  loading  is  on 
ESS.  No  compelling  reason  is  apparent  for  this  load¬ 
ing,  unless  divergent-production  tests,  too,  may  be¬ 
come  involved  with  evaluation  variance.  But  there  is 
generally  much  doubt  about  Number  Grouping,  for  it 
was  not  loaded  on  DSC  in  its  previous  factor -analytic 
investigation  and  appears  not  to  be  clearly  a  measure 
of  any  factor  found  in  the  analyses  in  which  it  has  been 
employed. 


EST  Evaluation  of  symbolic  transformations 

16.  Jumbled  Words  (EST)  .45 

10.  Decoding  (EST)  .  40  (.  39  ESC) 

39.  Typing  Errors  (EST)  .  31 

The  three  tests  designed  to  measure  EST  emerged 
on  the  EST  factor  with  unexpected  clarity.  The  con¬ 
struction  of  the  EST  tests  had  proved  to  be  most  difficult 
and  therefore  it  was  expected  that  EST  might  not  be 
found  in  this  analysis.  All  three  tests  designed  for 
EST,  and  only  those  three,  came  out  significantly  on 
EST.  All  three  tests  involve  the  use  of  words.  The 
strongest  for  EST  was  considered  to  be  a  sensitivity 
test,  the  other  two  were  considered  estimation  tests, 
but  one'of  them  is  not  far  behind  the  leading  test,  so 
there  is  no  conclusion  except  that  it  probably  does  not 
matter  which  type  is  used  for  EST  measurement. 

The  three  tests  differ  somewhat  in  terms  of  oper¬ 
ations  that  E  probably  performs  as  he  takes  the  tests. 
In  Jumbled  Words,  he  decides  whether  each  response 
word  could  have  come  from  the  given  word  merely  by 
rearrangement  of  the  letters  it  contains.  The  criterion 
is  in  terms  of  a  certain  invariance  or  of  element  identity 
under  the  transformation  of  changed  order. 

In  Decoding,  E  is  expected  to  apply  certain  rules, 
of  which  five  are  given,  in  coding  the  letters  ctf  each  of 
two  words  into  a  sequence  of  digits.  The  transform¬ 
ation  in  each  case  is  in  the  form  of  substitution  of  ele¬ 
ments  according  to  rules.  E  then  is  expected  to  decide 
which  coding  (transformation)  result  could  be  most  easily 
and  correctly  decoded.  The  difference  in  ease  and  cor¬ 
rectness  of  decoding  depends  upon  the  approach  of  the 
substitutions  for  a  word  to  univocality,  under  the  rules. 
That  is,  some  rules  pertain  to  a  smaller  class  of  let¬ 
ters.  For  example,  take  the  two  rules:  For  all  double 
letters  (oo,  gg,  etc.  ),  substitute  the  digit  1;  for  each 
single  consonant,  substitute  the  digit  5.  There  being 
more  alternatives  among  single  consonants  than  among 
occurrences  of  double  letters,  a  1  is  more  indicative 
in  a  coded  word  than  is  a  5,  for  which  uncertainty  in 
decoding  would  be  greater.  Assuming  that  E  follows 
this  principle,  the  criterion  is  the  preponderance  of 
high-probability  digits  in  each  word.  E  has  a  third 
category  of  response  that  he  uses  if  he  thinks  the  two 
coded  words  are  equally  near  univocality. 

In  Typing  Errors,  E  is  presented  with  what  he  is 
told  is  a  misspelled  word,  e.  g.  ,  "WORM.  "  His  alter¬ 
native  answers  are:”WARM,"  "WORN,"  and  "WARS." 
From  a  knowledge  of  the  arrangement  of  letters  on  the 
keyboard  of  a  typewriter,  which  transformation  in  spell  - 
ing  (substitution  of  one  letter  element)  has  most  prob¬ 
ably  occurred?  The  decision  boils  down  to  which  letter 
change  involves  two  letters  closest  together  on  the  key¬ 
board.  Incidentally,  the  keyboard  is  printed  on  each 
page  of  the  test  booklet.  A  correlation  of  scores  on 
this  test  with  amount  of  typing  experience  was  very  low, 
indicating  that  typing  experience  contributed  little  or 
nothing  to  variance  in  the  scores.  A  judgment  of  near¬ 
ness  of  two  letters  on  the  keyboard  may  involve  some 
other  ability,  perhaps  distance  estimation,  which  may 
account  for  the  minimally  significant  loading  on  EST 
for  this  test.  Whatever  EST  variance  there  is,  may 
come  from  the  comparison  of  alternative  transforma¬ 
tions  in  the  form  of  letter  substitutions. 
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To  summarize  these  comparisons,  the  transfor¬ 
mation  was  in  the  form  of  reordering  of  letters  in  one 
test  and  in  the  form  of  letter  substitutions  and  letter¬ 
digit  substitutions  in  the  other  two  tests.  The  criteria 
appear  to  be  identity  of  elements  in  spite  of  transfor¬ 
mation  in  one  test,  univocality  of  reference  in  the  coding 
test,  and  nearness  of  position  in  a  symbolic  system 
(keyboard)  in  the  third.  These  differences  suggest  that 
there  is  some  scope  in  kinds  of  transformations  and  in 
criteria  for  decision  involved  in  connection  with  factor 
EST. 

ESI  Evaluation  of  symbolic  implications 


2. 

Best  Letter  Set  {ESS) 

.47 

(.  34  NSI) 

1. 

Abbreviations  (ESI) 

.  45 

38. 

Symbol  Reasoning  (ESI) 

.  36 

(.  31  CMU) 

17. 

Letter  Problems  (ESI) 

.  35 

The  ESI  factor  is  defined  by  three  tests  designed 
as  measures  of  ESI,  but  is  led  by  a  test  designed  for 
ESS  that  is  not  loaded  on  that  factor.  The  most  obvious 
common  characteristic  of  the  two  leading  tests  is  that 
the  given  stimuli  are  sets  of  letters,  but  this  is  also 
true  of  many  other  tests  in  the  battery.  But  in  addition, 
the  item  formats  of  the  two  tests  are  similar  (see  Ap¬ 
pendix).  In  Abbreviations ,  a  sequence  of  three  or  four 
letters  is  presented  as  the  potential  abbreviation  of 
three  alternative,  familiar  words,  with  E  to  say  for 
which  word  the  abbreviation  best  stands  or  that  it  best 
implies.  In  Best  Letter  Set,  a  sequence  of  three  or 
four  letters  is  given  and  three  alternative  letter  sets 
of  the  same  length,  E  to  select  the  one  that  is  most 
similar  to  thp  given  set  by  virtue  of  common  properties. 
The  test  was  expected  to  be  a  measure  of  ESS  because 
it  was  thought  that  the  nearness  of  One  set  to  another 
in  terms  of  their  constitutions  would  be  a  matter  of 
comparing  systems  for  approach  to  identity  in  terms 
of  systemic  properties.  The  given  set  might  be  W  V  U, 
which  is  easily  recognized  as  three  consecutive  letters 
in  reverse  alphabetical  order.  ■  The  three  alternative 
answers  are:  D  C  A,  T  P  O,  and  Z  P  E.  None  of 
these  is  exactly  in  reverse  consecutive  order.  The 
first  two  are  in  reverse  order,  but  the  very  first  one 
comes  nearer  to  consecutive  order. 

It  is  not  very  clear  how  Best  Letter  Set  becomes 
an  implications  test  rather  than  a  systems  test.  It  is 
little  more  than  stating  the  obvious  to  say  that  the  given 
set  apparently  implies  the  best  alternative,  similarly 
to  the  way  in  which  a  letter  set  in  Abbreviations  implies 
a  longer  letter  set  in  the  form  of  a  real  word.  A  revised 
format  of  Best  Letter  Set,  giving  alternatives  that  either 
do  or  do  not  fit  the  principle  of  the  given  set  exactly, 
might  have  been  a  better  ESS  test.  But  there  would 
still  be  much  unanswered  about  underlying  reasons  for 
the  difference  in  factor  content  of  the  two  test  formats. 

Symbol  Reasoning  would  seem  to  be  an  ideal  type 
of  test  for  ESI,  for  it  requires  E  to  decide  whether  con¬ 
clusions,  expressed  in  symbolic  form,  can  or  cannot 
be  justifiably  drawn  from  other  symbolic  statements  in 
the  form  of  equations  or  inequalities.  In  essence,  this 
test  would  seem  parallel  to  the  verbal-syllogistic  test 
Logical  Reasoning,  which  has  more  recently  been  found 
to  measure  factor  EMI  (Nihira,  et  al.  ,  1964).  Just  as 
Togical  Reasoning  proved  to  be  somewhat  corriplex  fac- 
torially,  it  is  probable  that  Symbol  Reasoning  is  com¬ 
plex,  as  suggested  by  its  low  loading  of  .  36  and  its  sec¬ 
ondary  loading  of  .  31  on  CMU.  The  latter  component 


may  mean  that  there  is  some  verbalizing  of  the  prob¬ 
lems  or  there  is  some  difficulty  in  understanding  the 
instructions,  which  are  long  and  somewhat  involved.  A 
much  simpler  task  is  needed  in  order  to  achieve  a  more 
univocal  test.  One  would  have  to  take  care,  however, 
to  steer  away  from  factor  ESR,  for  which  a  similar 
test.  Symbol  Manipulation,  is  the  leading  test.  The 
parallel  test  Logical  Reasoning  shares  some  variance 
with  factor  EMR.  It  may  be  that  a  univocal  test  for 
ESI  lies  in  a  little  different  direction  than  a  test  like 
Symbol  Reasoning.  Of  the  tests  in  the  list  above.  Ab¬ 
breviations,  of  course,  appears  to  hold  the  most  prom¬ 
ise. 

Letter  Problems  holds  some  promise  of  univo¬ 
cality,  so  far  as  this  analysis  goes,  but  with  a  low  load¬ 
ing  of  .  35  and  a  reliability  of  ,  88,  there  is  considerable 
room  for  additional  common-factor  content.  Letter 
Problems  is  interesting  for  the  fact  that  it  was  developed 
by  analogy  to  Form  Reasoning  (see  the  Appendix),  which 
is  a  marker  test  for  factor,  NSI.  A  major  difference, 
which  should  not  be  factorially  significant,  is  that  Letter 
Problems  uses  letters  as  symbolic  elements  whereas 
Form  Reasoning  uses  geometric  forms  as  symbolic 
elements.  The  sigaificant difference  is  that  Form  Reason¬ 
ing  calls  for  inferences  or  conclusions  to  equations  of 
a  certain  type  whereas  Letter  Problems  asks  for  deci¬ 
sions  as  to  whether  a  problem  equation  is  solvable  (has 
a  determined  conclusion),  is  not  solvable,  or  is  solvable 
with  a  minor  change  in  the  problem.  It  might  be  said  that 
the  criterion  for  evaluation  is  solvability  or  the  possi¬ 
bility  of  valid  inferences  or  implications.  This  is  not 
strictly  a  matter  of  judging  the  value  or  identity  of  an 
implication,  as  such,  or  its  similarity  to  another  im¬ 
plication,  which  are  common  kinds  of  criteria  in  tests 
of  other  evaluation  factors.  But  if  a  new  kind  of  cri¬ 
terion  is  involved  and  is  crucial  to  the  loading  on  ESI, 
we  have  a  little' extension  of  connotation  of  evaluation 
abilities  and  the  evaluative  process  as  envisaged  from 
the  psychometric  approach. 

Interpretation  of  the  Experimental  Motivation  Factors 

The  last  two  factors  to  'oe  reported  are  each  con¬ 
fined  to  measures  of  a  particular  kind  and  represent  a 
failure  to  obtain  a  somewhat  general  motivation  factor 
thought  to  underlie  all  the  measures,  A  lack  of  corre¬ 
lation  among  some  of  the  four  hypothesized  motivation 
measures  made  it  impossible  to  force  the  measures 
onto  one  factor. 

RAT  Ratings 

49.  Rating  -  Booklet  Effort  (MOT)  .  55 
48.  Rating  -  Test  Liking  (MOT)  .  50 

The  tendency  for  self-ratings  to  reflect  bias,  and 
thus  intercorrelate  due  to  personal  consistency  in  bias, 
probably  accounts  for  most  of  the  specific  nature  of  this 
factor.  Similar  formats  of  rating  and  similar  tasks 
also  contribute  to  the  specific  nature.  Whatever  this 
factor  represents,  it  is  clear  that  it  is  not  related  to 
the  aptitude  measures  in  the  battery, 

PAD  Prediction-achievement  discrepancy 

51.  Prsdiction-Achlevament  Discrepancy  (MOT)  .56 


27 


This  singlet  is  the  second  example  of  failure  to 
obtain  the  hypothesized  motivation  factor.  The  least- 
squares  discrepancy  from  the  regression  of  academic 
achievement  on  DAT  s'ubscores  appears  to  be  not  so 
much  due  to  common-factor  motivational  aspects  of 
the  examinees  as  it  is  due  to  error  in  the  predictors, 
criterion,  or  other  traits  that  contribute  to  grade-point 
averages  but  are  not  measured  by  the  nine  DAT  sub¬ 
tests.  If  the  discrepancy  were  due  to  motivational  fac¬ 
tors,  the  scores  would  presumably  correlate  more 
highly  with  the  other  measures  that  presumably  contain 
some  motivational  variance. 

The  fact  that  certain  aptitude  scores  were  sub¬ 
tracted  to  obtain  the  PAD  score  is  the  reason  why  vari¬ 
able  51  correlates  low  or  negatively  with  all  the  aptitude 
measures,  a  number  of  which  are  related  to  the  DAT 
scores  that  were  subtracted.  Having  no  large  correla¬ 
tions  with  any  variate,  but  small  negative  ones  with 
most  of  them,  the  PAD  score  formed  a  singlet  factor  of 
its  own,  which  is  partially  or  wholly  spurious. 

DISCUSSION 

In  this  section,  the  experimental  findings  of  this 
study  are  considered  in  terms  of  how  they  bear  upon  the 
hypothesis  with  which  the  study  was  initiated,  both  in 
terms  of  how  they  indicate  the  continued  fruitfulness  of 
the  structure -of-intellect  theory  as  a  source  of  testable 
hypotheses  and  more  specifically  how  well  types  of  abili¬ 
ties  in  the  model  were  differentiated.  Relations  of  the 
findings  with  respect  to  symbolic-evaluation  factors  to 
the  very  limited  prior  indications  of  any  of  those  factors 
will  be  noted,  as  well  as  with  respect  to  the  findings  of 
a  recent  parallel  study  of  semantic-evaluation  abilities. 
General  light  that  is  thrown  upon  the  nature  of  evaluation 
as  a  category  of  abilities  and  as  a  distinct  kind  of  oper¬ 
ation  will  be  considered.  Ending  the  section  will  be  the 
usual  recommendations  regarding  tests  for  the  factors 
under  special  investigation. 


Relations  of  Results  to  the  Hypothesis 

Since  so  many  factors  pertaining  to  the  structure- 
of-intellect  model  were  under  examination  along  with 
the  symbolic -evaluation  set  of  factors,  a  somewhat  gen¬ 
eral  consideration  can  be  given  to  how  well  factor- 
analytic  operations  of  the  kinds  used  can  differentiate 
factors  in  different  categories.  Abilities  will  be  dis¬ 
tinguished  in  terms  of  operations,  in  terms  of  contents, 
then  in  terms  of  products,  with  special  emphasis  upon 
the  experimental  evaluation  factors. 

Relation  to  Hypothesized  Operations 

All  of  the  five  kinds  of  operation  were  involved  in 
this  study,  but  obviously  not  equally  so.  There  were 
live  factors  of  symbolic  cognition,  the  sixth  one  (CST) 
not  yet  having  been  demonstrated,  and  one  of  semantic 
cognition  (CMU);  one  memory  factor  (MSI);  one  diver- 
gent-production  factor  (DSC);  three  convergent-production 
factors  (NSS,  NST,  and  NSl);  and  seven  evaluation  fac¬ 
tors,  one  figural  (EFU)  and  the  six  symbolic-evaluation 
factors  of  special  interest.  The  factors  outside  the 
last-mentioned  six  were  reference  factors,  with  the 
usual  concern  lest  some  of  the  new  factors  not  be  shown 


to  be  distinct  from  parallel  factors  in  other  operation 
categories  or  lest  some  of  the  variances  of  new  experi¬ 
mental  tests  not  be  well  accounted  for.  The  greatest 
concern  was  regarding  the  demonstration  that  the 
symbolic-evaluation  abilities  be  differentiated  from  the 
parallel  cognition  factors.  In  constructing  evaluation 
tests,  it  is  not  easy  to  be  sure  that  cognitive  variance 
has  been  ruled  out  by  the  experimental  controls  in  the 
test  conditions,  or  even  that  cognitive  variance  may  not 
dominate  the  test.  Other  reference  factors  were  brought 
in  because  new  experimental  tests  were  suspected  of 
involving  still  other  operation  variances. 

The  general  outcome  was  that  six  symbolic- 
evaluation  abilities  could  be  demonstrated  as  distinct 
from  abilities  of  cognition,  memory,  and  divergent 
and  convergent  production.  How  well  this  was  accom¬ 
plished  is  indicated  by  the  fact  that  22  of  25  experimental 
symbolic-evaluation  tests  cohered  with  their  respective 
factors  in  that  category. 

Another,  more  general,  indicator  is  the  frequency 
with  which  tests  designed  for  the  various  operations 
have  significant  loadings  on  factors  in  those  operation 
categories,  although  this  does  not  tell  the  whole  story. 
On  the  factors  of  cognition  (CSU,  CSC,  CSR,  CSS,  CSI, 
and  CMU)  there  were  15  such  loadings  for  cognition 
tests,  two  for  convergent-production  tests,  and  seven 
for  evaluation  tests.  As  feared,  there  was  greatest 
danger  that  evaluation  tests  would  be  involved  with  cog¬ 
nition  variances.  But  the  picture  is  somewhat  better 
when  we  consider  that  there  were  only  13  tests  thought 
in  advance  to  be  cognitive  (two  had  two  loadings  each  on 
cognitive  factors,  making  15  loadings)  and  there  were 
27  tests  slated  for  evaluation. 

With  memory  abilities  and  divergent-production 
abilities  represented  by  one  factor  each,  it  is  not  pos¬ 
sible  to  extract  anything  by  way  of  generalization  about 
those  two  operation  categories.  As  to  tests  loaded  sig¬ 
nificantly  on  the  three  convergent-production  factors  ■ 
(NSS,  NST,  and  NSI),  all  six  of  the  convergent-production 
tests  had  significant  loadings  on  their  respective  fac¬ 
tors,  There  were  also  three  from  the  category  of  cog¬ 
nition,  one  from  memory,  and  three  from  evaluation. 

On  the  seven  evaluation  factors  there  were  28  sig¬ 
nificant  loadings  for  evaluation  tests,  two  for  cognition 
tests,  one  for  a  divergent-production  test,  and  two  for 
a  convergent-production  test.  It  would  appear  to  be 
easier  to  keep  evaluation  out  of  cognition  tests  than  to 
keep  cognition  out  of  evaluation  tests,  although  this  de¬ 
pends  somewhat  upon  where  the  rotations  happen  to  go 
in  a  particular  analysis.  At  any  rate,  from  another 
point  of  view  the  six  experimental  evaluation  factors 
appear  to  be  well  differentiated  from  other  operation 
factors.  Although  one  might  expect  evaluation  to  play 
roles  in  connection  with  convergent-production  tests,  in 
which  responses  must  be  narrowed  down  to  a  single  one 
for  each  item,  as  far  as  can  be  seen  there  is  little  rea¬ 
son  to  doubt  the  distinctness  of  those  two  operation 
categories. 

Relation  to  Hypothesized  Contei^ 

There  being  only  one  figural  factor  and  one  se¬ 
mantic  factor  in  the  analysis,  it  is  not  easy  to  derive 
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much  in  the  way  of  a  test  of  the  hypothesis  of  three  dis¬ 
tinct  kinds  of  abilities  as  to  content  or  kind  of  inform¬ 
ation,  But  in  the  graphic  rotations,  it  was  very  apparent 
that  the  figural  and  semantic  factors  were  separate  from 
the  remaining  symbolic  factors.  Of  the  16  symbolic 
factors,  none  had  on  them  any  significant  loadings  from 
the  two  figural-evaluation  tests  and  two  had  significant 
but  small  loadings,  one  each,  from  two  semantic  tests. 
For  the  one  semantic  factor  (CMU),  four  of  the  seven 
significant  loadings  were  in  symbolic  tests.  The  latter 
kind  of  result  has  been  rationalized  on  the  basis  of  CMU 
variance  contributed  from  variations  in  verbal  under¬ 
standing  of  the  instructions  and/or  from  E's  verbalizing 
his  work  on  the  items  of  the  four  tests. 

Another  point  regarding  kind  of  information  should 
be  mentioned.  This  pertains  to  what  kinds  of  test  ma¬ 
terials  can  be  used'  as  stimuli  for  eliciting  symbolic 
information.  The  results  of  this  study  demonstrate 
quite  clearly  that  symbols  can  be  in  the  form  of  num¬ 
bers,  letters,  syllables,  words,  or  even  geometric 
forms  serving  as  signs  or  tokens.  No  experimental 
(symbolic-evaluation)  factor  is  defined  solely  by  tests 
employing  one  kind  of  symbol  to  the  exclusion  of  the 
others.  Instead,  most  of  the  factors  are  defined  by  all 
three  types  of  symbols  or  by  tests  employing  combi¬ 
nations  of  the  types. 

Relation  to  Hypothesized  Products 

Among  the  six  hypothesized  symbolic-evaluation 
factors;  there  is  very  clear  delineation.  From  inspec¬ 
tion  of  the  significant  loadings,  there  appears  to  be  very 
little  sign  of  obliqueness  between  any  two  of  the  six 
factors.  The  factors  ESU,  ESS,  and  EST  show  no  sig¬ 
nificant  loadings  from  tests  of  other  symbolic-evaluation 
factors.  Factors  ESR  and  ESI  show  one  such  loading 

each - ESR  for  an  ESU  test  with  a  minimal  loading  of 

.  31,  but  ESI  with  an  ESS  test  taking  the  lead  for  ESI 
with  a  loading  of  .  47i  But  neither  result  throws  any 
doubt  upon  the  distinctness  of  factors.  Factor  ESC 
shows  the  most  involvement  in  tests  for  other  ES  fac¬ 
tors,  having  its  highest  loading  of  ,  54  from  an  (expected) 
ESR  test  and  a  very  small  loading  of  .  33  for  another 
(actual)  ESR  test,  ESR  does  not  reciprocate  by  having 
any  significant  loadings  for  ESC  tests.  Other  "foreign" 
tests  loaded  on  ESC  are  an  ESU  test  with  .  33  and  an 
EST  test  with  .  39.  Other  sets  of  six  tests  in  a  column 
of  the  model  have  shown  less  clarity  of  separation  than 
is  indicated  among  these  six  evaluation  factors. 


Relations  to  Previous  Results  on  Evaluation  Factors 


As  mentioned  in  the  discussion  of  the  previous 
literature,  there  has  been  very  little  clear  precedent 
for  any  of  the  six  experimental  factors  of  this  study. 

There  were  tenuous  evidences  for  only  two  of  them - 

factors  ESU  and  ESR.  As  reported  by  French  (1951), 
Thurstone,  Coombs,  Bechtoldt,  and  others  sometimes 
found  tests  calling  for  the  identification  of  letters  or  the 
matching  of  letters,  and  even  combinations  of  letters, 
sometimes  to  appear  loaded  on  perceptual  speed  (now 
identified  as  EFU)  and  sometimes  suggesting  a  separate 
factor  (a  possible  ESU),  The  trouble  was  that  a  dis¬ 
tinction  between  figural  and  symbolic  information  had 
not  been  recognized  and  not  enough  tests  of  the  two  kinds 


were  always  analyzed  together.  Letters  taken  singly 
are  likely  to  serve  as  material  for  figural  information, 
and  cancellation  tests  and  the  Scattered  X' s  test  are 
more  definitely  more  figural  than  symbolic.  The  Letter 
A  test,  which  is  essentially  an  alternate  form  of  Letter 
"U"  used  in  this  study,  usually  had  a  weak  EFU  loading 
and  tended  to  help  suggest  the  second  factor.  This  is 
the  only  link  between  a  possibly  earlier  ESU  and  the 
factor  demonstrated  in  this  study. 

The  possible  earlier  sign  of  a  factor  ESR  came  out 
of  analyses  at  the  Aptitudes  Research  Project  at  the 
University  of  Southern  California,  in  which  Symbol  Mar 
nipulation  tended  to  lead  on  a  factor  tentatively  placed 
in  the  unique  spot  ESR  because  Symbol  Manipulation  is 
a  kind  of  symbolic-syllogism  test,  and  a  verbal- 
syllogism  test  consistently  defined  a  factor  called  "logi¬ 
cal  evaluation,  "  and  in  due  time  it  was  placed  in  the 
parallel  cell  of  EMR.  Symbol  Manipulation  also  some¬ 
times  had  significant  loadings  on  the  logical-evaluation 
factor,  which  helped  to  support  its  factor's  claim  to  the 
cell  ESR.  From  the  liberal  support  given  to  the  inter¬ 
pretation  of  the  ESR  factor  in  this  study,  it  appears  that 
the  symbol  manipulation  factor  was  properly  placed,  and 
the  test  by  that  name  still  leads  on  the  factor  by  a  good 
margin. 

A  comparison  of  the  results  of  this  study  with 
those  from  the  recently  reported  analysis  of  semantic - 
evaluation  factors  (Nihira,  et  al.  ,  1964)  is  of  some  in¬ 
terest.  Although  many  of  the  tests  are  different  in  form, 
there  are  naturally  some  parallels.  For  example,  tests 
of  ESC  were  direct  translations  of  the  EMC  tests  into 
symbolic  content.  Best  Number  Pairs  was  an  adaptation 
of  Best  Word  Pairs.  The  two  performed  very  similarly 
in  that  both  went  rather  on  their  respective  cognition 
parallels,  factors  CSC  and  CMC.  In  both  cases  it  can 
be  pointed  out  that  for  measurement  of  evaluation  rather 
than  cognition,  there  is  a  need  to  state  explicitly  the 
criteria  upon  which  judgments  are  to  be  made,  and  pos¬ 
sibly  provide  models  for  comparison  or  to  describe 
them,  as  in  Double  Descriptions  and  Letter  "U". 

Another  parallel  between  results  in  the  two  studies 
concerns  the  implications  factors  ESI  and  EMI.  The 
two  parallel  factors  are  defined  in  part  by  tests  employ¬ 
ing  syllogistic  items.  For  ESI,  Symbol  Reasoning  em¬ 
ploys  symbolic-syllogistic  items  and  for  EhU,  l-ogical 
Reasoning  employs  the  more  usual  verbal- syllogistic 
items.  Both  have  some  tendency  toward  factorial  com¬ 
plexity.  But  whereas  logical  Reasoning  has  a  signifi¬ 
cant  loading  on  EMR,  for  which  it  was  formerly  believed 
to  be  one  of  the  best  measures.  Symbol  Reasoning  does 
not  have  a  parallel  significant  loading  on  ESR.  This 
may  be  because  the  given  premises  in  the  latter  do  not 
contain  simple  relations. 

Other  parallels  between  tests  could  be  pointed  out, 
but  this  could  become  tedious.  One  more  will  be  men¬ 
tioned,  however,  a  case  in  which  the  appearance  of  test 
similarity  is  not  so  great;  but  the  psychological  simi¬ 
larity  is  good.  This  case  is  for  the  test  Letter  "U"  for 
factor  ESU  and  the  test  Double  Descriptions  for  factor 
EMU.  Both  involve  specifying  properties  of  the  kind 
of  unit  to  be  used  as  the  "model”;  i.  e.  ,  its  class  is 
specified.  In  Letter  "U",  E  is  to  decide  which  words 
satisfy  the  specification  "words  containing  the  letter 
'U'.  "  In  Double  Descriptions.  E  is  to  decide  which 
object  most  nearly  satisfies  two  properties,  such  as 
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both  white  and  edible.  Deciding  whether  or  how  well 
given  objects  fit  the  specifications  of  the  class  is  the 
crucial  part  of  either  task. 

But  one  can  also  cite  instances  in  which  seemingly 
parallel  tests  do  not  both  measure  two  parallel  evalu¬ 
ation  factors,  even  factors  ESU  and  EMU.  The  test 
Symbol  Identities  is  the  leading  measure  of  ESU.  It 
calls  for  decisions  as  to  whether  two  letter  or  number 
sets  are  identical  in  every  detail.  Two  tests  intended 
for  EMU  emphasized,  not  identities  of  word  meanings 
but  very  near  similarities  of  word  meanings  (Nihira, 
et  al,  ,  1964).  One  was  a  matter  of  deciding  which  syno¬ 
nym  comes  nearest  in  meaning  to  a  given  word,  when 
words  are  given  out  of  context,  and  the  other  put  the  stem 
word  for  the  item  within  the  context  of  a  sentence.  Both 
tests  failed  to  have  significant  relations  with  EMU  and 
had  significant  loadings  on  CMU  instead.  Such  dis¬ 
crepancies  call  for  further  investigation.  It  would 
appear  that  some  of  the  same  testing  principles  cannot 
be  effectively  applied  as  we  change  kind  of  content  or 
kind  of  product  within  an  area  of  evaluation. 

New  Light  on  the  Properties  of  Evaluation 

What  can  it  be  said  that  the  study  has  done  for 
further  elucidation  of  the  concept  of  the  operation  of 
evaluation?  What  aspects  of  the  concept  possibly  need 
changing  as  a  result  of  the  new  information  about  factors 
and  their  tests,  and  what  new  features  come  into  the  pic¬ 
ture?  We  shall  consider  the  answers  to  these  general 
questions  more  specifically  in  terms  of  (1)  the  kinds  of 
judgments  that  belong  in  the  picture  of  evaluation  as 
defined  by  the  tests;  (2)  what  kinds  of  criteria  for  judg¬ 
ment  are  pertinent  to  measurement  of  the  evaluation 
abilities;  (3)  whether  remembered  information  can  be 
utilized  in  evaluation;  and  (4)  whether  there  are  any  new 
restrictions  to  be  placed  on  the  concept.  The  answers 
to  these  issues  rest  upon  the  kinds  of  tests  that  serve 
to  measure  their  evaluation  factors  well  and  those  that 
do  not,  when  all  have  been  hypothesized  to  do  so. 


Kinds  of  Judgment 

Much  was  said  from  one  place  to  another  in  this 
report  about  the  two  classes  of  tests;  sensitivity  vs. 
estimation.  More  fully  spelled  out,  these  terms  mean 
sensitivity  to  error  or  discrepancy  on  the  one  hand  and, 
on  the  other,  judgment  of  relative  nearness  of  a  number 
of  items  of  information  (for  any  kind  of  product)  to  a 
kind  of  model  item  of  information  on  the  same  continuum. 
In  more  operational  terms,  the  contrast  rhay  be  stated 
in  terms  of  "absolute"  vs,  "relative"  kinds  of  judgments, 
as  in  psychophysics. 

It  should  be  abundantly  clear,  from  the  way  in 
which  both  sensitivity  and  estimation  types  of  tests  are 
commonly  related  significantly  to  the  same  factors, that 
both  kinds  of  judgments  apply.  If  we  compare  evaluation 
tests  from  the  two  categories  that  clearly  involve  abso¬ 
lute  vs.  relative  judgments,  however,  we  find  that  the 
11  requiring  absolute  judgments  tend  to  have  slightly 
stronger  loadings  on  the  factors  for  which  they  were 
intended  than  the  12  requiring  relative  judgments,  the 
average  difference  being  about  .  07.  Four  of  the  12  tests 
involving  relative  judgments  had  higher  loadings  on  fac¬ 
tors  for  which  they  were  not  intended,  only  one  of  the 


four  having  significant  loading  on  its  intended  factor,  the 
other  such  loadings  being  .  01,  ,  09,  and  .  20.  This  com¬ 
pares  with  two  of  the  11  tests  involving  absolute  judg¬ 
ments  that  had  higher  loadings  on  factors  for  which  they 
were  not  intended,  with  loadings  of  .  35  and  .  29  on  fac¬ 
tors  for  which  they  were  intended. 

Although  these  differences  are  in  all  probability 
not  significant,  they  suggest  a  trend.  On  the  whole, 
tests  requiring  absolute  judgments  are  operationally 
simpler,  which  should  be  more  favorable  for  univocality. 
The  best  examples  of  such  simple  tests  are  Perceptual 
Speed  and  Identical  Forms,  for  factor  EFU,  and  Symbol 
Identities  and  Letter  "U",  for  factor  ESU. 

Kinds  of  Criteria 

The  most  common  criteria  employed  in  e'.  aluation 
tests  have  been  identity  vs.  non-identity  and  consistency 
vs.  inconsistency.  Other  kinds  of  criteria  have  been 
mentioned  in  connection  with  various  tests  and  for  dif-, 
ferent  factors.  Some  examples  are:  fitness  for  class 
membership  (Letter  "U");  relative  familiarity  (Familiar 
Letter  Combinations);  relative  similarity  (Series  Rela¬ 
tions);  conformity  to  principles  (Correct  Number  Series, 
Correct  Letter  Orders);  relative  probability  (Decoding ); 
and  solvability  of  problems  (Letter  Problems).  Such  a 
variety  of  criteria  can  possibly  be  brought  under  more 
abstract  and  more  general  criterion  categories,  since 
such  terms  as  "identity,"  "sirr.ilarity,  "  and  "conformity" 
suggest  that  the  criteria  tend  to  be  logical  in  nature  and 
that  they  represent  continua  of  one  kind  or  another. 

Perceived  vs.  Remembered  Models 

There  were  two  tests  designed  for  factor  ESU  that 
failed  to  function  as  expected  — Correct  Spelling  and 
Familiar  Letter  Combinations.  A  reason  offered  for 
the  failures  was  to  the  effect  that  in  both  instances  the 
examinee  has  to  draw  upon  his  memory  storage  for 
something  with  which  to  compare  the  given  (jjerceived) 
item  of  information.  This  suggested  the  generalization 
that  in  evaluation  activities  the  comparisons  must  be 
between  items  of  information  both  present  to  cognition, 
if  not  to  perception,  at  the  moment.  No  other  evaluation 
tests  offered  the  possibility  of  finding  clear  evidence  on 
this  point,  thus  leaving  the  question  open.  From  a  priori 
thinking,  it  would  seem  that  in  everyday  life  we  are  per¬ 
petually  comparing  presently  cognized  information  with 
remembered  information.  There  would  seem  to  be  good 
possibilities  for  testing  the  hypothesis  experimentally. 

The  Scope  of  Evaluation 

The  scope  of  processes  under  the  heading  of  evalu¬ 
ation  is  indicated  somewhat  by  the  variety  of  criteria 
that  may  be  involved.  The  discussion  of  this  topic 
above  revealed  something  of  the  apparent  variety  of 
criteria  that  is  represented  in  the  experimental  tests. 
But  it  was  suggested  that  such  criteria  may  be  limited 
to  the  general  logical  category,  which  would  rule  out  of 
consideration  criteria  involving  ethical  and  esthetic 
values.  There  is  no  doubt  that  such  values  exist  and 
such  areas  of  judgment  call  for  evaluative  operations. 
At  the  present,  they  do  not  seem  to  fit  into  the  structure - 
of-intellect  model.  Perhaps  they  call  for  two  complete 
additional  sets  of  evaluative  abilities  or  processes, 
whether  parallel  to  the  present  theoretical  set  or  not. 
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As  to  the  definition  of  evaluation  itself,  the  kind 
of  restriction  just  discussed  suggests  that  it  is  going 
too  far  to  say  that  evaluation  is  a  matter  of  reaching 
decisions  regarding  goal  satisfaction.  Kinds  of  goals 
are  much  too  numerous,  and  satisfaction  in  terms  of 
logical  criteria  cannot  cover  all  cases.  In  defining  the 
restricted  kinds  of  evaluation  represented  in  the  struc¬ 
ture  of  intellect,  it  would  seem  desirable  to  eliminate 
reference  to  "goal  satisfaction.  " 

As  a  general  impression,  from  consideration  of 
the  experimental  tests  and  their  factors  in  this  study, 
the  importance  of  an  act  of  comparison  seems  to  stand 
out.  This  observation  was  also  made  by  Hertzka  (1953) 
when  he  said,  "The  core  of  this  definition  is  the  concept 
of  comparison.  "  The  following  current  definition  of 
evaluation  can  be  suggested;  Evaluation  is  a  process  of 
comparing  a  product  of  information  with  known  informa¬ 
tion  according  to  logical  criteria,  reaching  a  decision 
concerning  criterion  satisfaction. 

Tests  Recommended  for  Symbolic -Evaluation  Factors 
The  following  tests  demonstrate  a  sufficient  degree 


of  univocality  and  saturation  of  variance  to  merit  being 
recommended  as  tests  of  their  respective  factors  in 
future  research. 

ESU  36.  Symbol  Identities 

19.  Letter  "U” 

ESC 

32. 

3. 

Sign  Changes  11 

Best  Number  Class 

ESR 

37. 

27. 

Symbol  Manipulation 

Related  Words  I 

ESS 

30. 

41. 

Series  Relations 

Way-Out  Numbers 

EST 

16. 

Jumbled  Words 

ESI 

1. 

Abbreviations 

The  authors  do  not  mean  to  imply  that  these  tests 
are  to  be  accepted  unquestionedly  as  the  ultimate  mea¬ 
sures  of  their  respective  factors,  but  rather  that  these 
tests  appear  to  be  the  best  measures  available  at  present. 
All  the  tests  await  further  refinement  and  analysis  before 
it  can  be  said  with  certainty  that  any  one  of  them  is  the 
best  possible  measure  of  its  factor. 

SUMMARY 


The  study  was  designed  to  test  the  implications  of, 
and  extend  the  empirical  foundations  underlying  the 
structure-of-intellect  model.  The  six  hypothesized 
abilities  of  symbolic  evaluation  were  selected  for  inves¬ 
tigation.  Abilities  in  this  category  appear  to  be  impor¬ 
tant  for  mathematical  thinking. 

Five  major  objectives  of  the  study  were:  (1) 
Demonstration  of  the  six  symbolic -evaluation  factors 
hypothesized  by  the  structure-of-intellect  model. 

(2)  Investigation  of  what  mental  processesare  evaluative. 

(3)  Investigation  of  what  materials  are  processed  sym¬ 
bolically.  (4)  Clarification  of  selected  known  factors 


of  the  model  through  the  use  of  shortened  and  refined 
tests.  (5)  Evaluation  of  the  theoretical  model  as  a 
basis  for  predicting  still-unknown  factors  and  for  de¬ 
veloping  tests  of  these  factors. 

The  six  experimental  factors  were;  evaluation  of 
symbolic  units  (ESU),  evaluation  of  symbolic  classes 
(ESC),  evaluation  of  symbolic  relations  (ESR),  evaluation 
of  symbolic  systems  (ESS),  evaluation  of  symbolic  trans¬ 
formations  (EST),  and  evaluation  of  symbolic  implica¬ 
tions  (ESI). 

In  order  to  demonstrate  the  distinctness  of  the 
hypothesized  factors  from  those  already  known,  12 
reference  factors,  previously  confirmed,  were  analyzed 
as  experimental  controls:  cognition  of  symbolic  units 
(CSU),  cognition  of  symbolic  classes  (CSC),  cognition 
of  symbolic  relations  (CSR),  cognitipn  of  symbolic  sys¬ 
tems  (CSS),  cognition  of  symbolic  implications  (CSI), 
cognition  of  semantic  units  (CMU),  memory  for  symbolic 
implications  (MSI),  divergent  production  of  symbolic 
classes  (DSC),  convergent  production  of  symbolic  sys¬ 
tems  (NSS),  convergent  production  of  symbolic  trans¬ 
formations  (NST),  convergent  production  of  symbolic 
implications  (NSI),  and  evaluation  of  figural  units  (EFU). 

A  thirteenth  factor,  not  formally  hypothesized  by 
the  structure-of-intellect  model,  was  included  in  order 
to  determine  its  possible  relationships  to  performance 
on  intellectual  tests:  motivation  to  succeed  (MOT). 

In  order  to  accomplish  the  experimental  objectives, 
53  test  measures  were  constructed,  adapted,  or  selected 
to  compose  an  eight-hour  battery.  Twenty-five  tests 
were  designed  to  measure  the  six  experimental  factors, 
the  remaining  tests  were  adapted  marker  tests  for  ref¬ 
erence  factors,  ratings  designed  to  reflect  motivational 
level,  and  published  tests  from  school  records. 

The  battery  was  administered  to  225  high-school 
seniors  in  a  Southern  California  community.  The  scores 
were  intercorrelated  and  factor  analyzed.  The  twenty 
principal-axis  factors  were  rotated  both  graphically  and 
analytically,  observing  the  criteria  of  simple  structure, 
positive  manifold,  and  psychological  meaningfulness. 

All  six  hypothesized  factors  emerged,  defined 
largely  by  the  tests  designed  to  measure  them.  Twelve 
reference  factors  were  isolated,  indicating  that  the 
shortened,  refined  tests  remained  relatively  factorially 
invariant  over  several  analyses  and  major  revisions, 
A  general  motivation  factor  failed  to  emerge.  Instead, 
a  rating  doublet  and  prediction-achievement-discrepancy 
singlet  were  found. 

Among  the  six  experimental  factors,  ESU  was 
clearly  defined  as  a  factor  that  could  be  described  as 
clerical  speed  and  accuracy,  the  ability  to  judge  rapidly 
symbolic  material  in  terms  of  identity  or  error.  The 
ESC  factor  was  least  clearcut,  but  it  involved  sensitivity 
to  class  properties.  ESR  was  clearly  isolated  as  the 
ability  to  make  choices  among  symbolic  relationships 
on  the  bases  of  identity  and  consistency.  The  ESS  factor 
appeared  to  involve  the  estimation  of  similarity  among 
series.  The  clear  isolation  of  EST  defined  it  as  the 
ability  of  sensitivity  to  the  identical  nature  of  rearrange¬ 
ments  and  substitutions  of  letters  within  words.  The 
ability  to  judge  the  consistency  or  probability  of  impli¬ 
cations  from  symbolic  material  was  represented  by  the 
ESI  factor. 
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From  the  tendency  of  the  hypothesized  factors  to 
be  defined  by  tests  involving  sensitivity  or  estimation 
(absolute  or  relative  judgments),  it  is  concluded  that  the 
two  are  to  be  considered  functionally  equivalent  in  evalu¬ 
ative  operations. 

This  study  further  confirmed  the  established  find¬ 
ing  that  numbers,  letters,  and  words  can  be  processed 
as  symbolic  information. 

Relatively  clear  separation  among  abilities  accord¬ 
ing  to  operation,  content,  and  product  was  interpreted 
as  substantially  contributing  to  the  value  of  the  unified 
theoretical  model  of  the  structure  of  intellect  as  a 
hypothfetico-deductive  theory  for  generating  predictions 
concerning  individual  differences  in  intellectual  func¬ 
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APPENDIX 


DESCRIPTION  OF  TESTS 


1-  Abbreviations  -  ESIOIB.  Choose  one  of  three  alternative  words  that  a  given  abbreviation  best  implies. 


Sample  Item;  A.  crescent  Answer:  C 

crnt  B.  coronation 
C.  current 


Score:  Number  right  minus  one-half  number  wrong. 
Parts;  2;  items  per  part:  15;  working  time:  8  min¬ 
utes. 


2.  Best  Letter  Set  -  ESS03A.  Choose  one  of  three  alternative  letter  sets  that  is  most  like  the  given  set. 

Sample  Item:  A.  JFI  Answer:  C  (begins  with  a  vowel) 

EKN  B.  PAQ 

C.  YBT  Score:  Number  right  minus  one-half  number  wrong.  Parts;  2;  items 

per  part:  15;  working  time:  10  minutes. 

3.  Best  Number  Class  -  ESCOIA.  Judge  into  which  class  a  given  number  fits  so  as  to  receive  the  most  possible 
polnfi"!  The  classes  and  their  points  are;  EVEN  MULTIPLES  -  1  point;  ODD  MULTIPLES  -  2  points;  SQUARES  - 
3  points;  PRIMES  -  4  points. 


Sample  Item;  100  Answer;  Class  of  SQUARES  Score;  Number  right  minus  one-third  number  wrong. 

Parts:  2;  items  per  part:  15;  working  time;  6  minutes. 


4.  Best  Number  Pairs  -  ESC02A.  Choose  one  of  three  number  pairs  that  makes  the  best  class.  In  order,  from  best 
to  worst,  the  classes  are:  perfect  squares,  inultiples,  odd-or-evens,  and  no  property  in  common. 


Sample  Item: 

A. 

6  - 

4 

^  Answer:  B  (perfect  squares) 

B. 

4  - 

9 

C. 

9  - 

6 

Score:  Number,  right  minus  one-half  number  wrong, 
per  part;  15;  working  time;  8  minutes. 

Parts;  i 

1;  items 

5.  Camouflaged  Words  -  NSTOIA.  Find  within  a  meaningful  sentence  a  group  of  consecutive  letters  that,  in  the  given 
ordeTi  spell  the  name  of  a  sport  or  game. 


Sample  Item: 


COW  A 


IS  NOT  A  SOLDIERLY  ATTRIBUTE 


Answer:  DICE 


Score:  Number  of  correct  responses,  one  camouflaged  word  per  item.  Parts;  1;  items:  15;  working  time:  6  minutes.. 


6.  Circle  Reasoning^  -  CSSOID.  Discover  the  principle  by  which  one  circle  is  blackened  in  each  of  four  rows  of  cir¬ 
cle  slmSTHasKeT!  Spply  the  rule  to  the  fifth  row. 


Sample  Item: 


0*0-00-000-00  -  O 

-0-*-0000-000 - 

o-*o-oo-ooo-ooo 

0*-00-0 - 0000-0 

O-J^O-O-OO-OOOO- 


Answer:  The  second  circle  from  the  left  is  blackened. 
Score;  Number  of  correct  responses.  Parts;  1;  items; 
14;  working  time:  8  minutes. 


7.  Correct  Letter  Orders  -  ESS04A.  Judge  whether  or  not  the  order  of  given  letters  follows  a  given  rule. 

Sample  Items:  Rule  -  Alternate  letters  in  the  alphabet  (skipping  one). 

I.  M  O  Q  S  U  W  Answer:  I  follows  the  rule;  II  does  not  follow  the  rule. 

II.  P  R  S  U  W  Y 

Score;  Number  right  minus  number  wrong.  Parts;  2;  items  per  part;  20;  working  time;  6  minutes. 

8.  Correct  Number  Series  -  ESS05A.  Judge  whether  or  not  the  order  of  given  numbers  follows  a  given  rule. 

Sample  Items;  Rule  -  Alternately  add-l,  multiply  by  3. 

I.  2  3  9  10  30  31  Answer;  I  follows  the  rule;  II  does  not  follow  the  rule. 

II.  4  5  15  16  49  50 

Score:  Numifter  right  minus  number  wrong.  Parts:  2;  items  per  part:  20;  working  time:  6  minutes. 

9.  Correct  Spelling  -  ESU04A.  Judge  whether  or  not  given  words  are  spelled  correctly. 

Sample  Items:  I.  experience  Answer:  I  correct;  II  incorrect. 

II.  seperate 

Score:  Number  right  minus  number  wrong.  Parts:  2;  items  per  part:  60;  working  time:  6  minutes. 

^Adapted  with  permission  from  a  test  by  Robert  Blakey. 
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10.  Decoding  -  ESTOIA.  Choose  the  word  that  would  be  easier  to  decode  it  coded.  The  code  is  based  upon  probabili¬ 
ties  ot  letter  occurrences  and  is  a  highly  ambiguous  code. 


Code:  All  double  letters  (oo,  gg,  etc.)  are  1. 

All  pairs  ot  vowels  (ea,  ou,  etc.)  are  2. 

All  pairs  ot  consonants  (bl,  sh,  etc.)  are  3. 
All  vowels  (a,  e,  i.  o,  u.  y)  are  4. 

All  consonants  (b,  g,  p,  etc. )  are  S. 


Sample  Item:  A. 

B. 

C. 


little 

blood 

both  words 


Answer:  B 


Score:  Number  right  minus  one  - hal t  numbe r  wrong.  Parts:  2; 
items  per  part:  15;  working  time:  8  minutes. 


11.  Derivations  -ESU08A.  Rapidly  judge  whether  words  can  be  derived  trom  agiven  word  by  using  some  ot  its  letters. 
Sample  Items:  Given:  PROCRASTINATE 


I.  trap  Answer:  I  can  be  derived;  II  and  III  cannot. 

II.  crust 

111.  percent  Score:  Number  right  minus  number  wrong.  Parts:  3;  items 

per  part:  50;  working  time:  9  minutes. 


12.  Disemvowelled  Words^  -  CSU04B.  Recognize  tamiliar  words  with  dashes  in  place  ot  vowels;  then  complete  the 
words  by  writing  the  vowels. 


Sample  Item;  m  t  1  t  Answer:  mutilate  Score:  Number  ot  correctly  completed  words.  Parts:  1; 

items;  25;  working  time;  5  minutes. 

13.  Familiar  Letter  Combinations  -  ESU05A.  Choose  one  of  two  given  letter  combinations  that  is  the  more  tamiliar 
syllable.  ' 


Sample  Item:  A.  1  o  y 

B.  n  i  s 


Answer:  A  Score;  Number  right  minus  number  wrong.  Parts;  2;  items 

per  part;  20;  working  time:  6  minutes. 


14.  Form  Reasoning^  -  NSI02C. 


From  the  table,  find  the  form  that  is  equal  to  the  three  given  forms. 


Table; 


☆  o=n 

o  ☆  =  □ 

o  o  =  ☆ 

□  0  =  A 

on  =  A 

c  o  =  0 

oc  =  o 

☆  0  =  0 

<>•☆ =o 

00  =  0 

00=0 

□  o  =  c 

OD-C 

OC=  8 

C  0=  8 

Sample  Item: 

☆  o  o 

Answer;  yy 


☆  □  C  A 


Score;  Number  right  minus  one -third 
number  wrong.  Parts;  1;  items:  20; 
working  time;  4  minutes. 


15.  Identical  Forms'^ 


EFU02A.  Find  one  of  five  figures  that  is  exactly  the  same  as  the  given  figure. 


Sample  Items; 


B 

cy  13  B9  (±1  □ 

/ 

4 

O  ♦  ♦ 

3 

^  ^  ^  dS 

4 

®  ®  ®  ® 

2 

TO 

5 

Score:  Number  right  minus  one-fourth 
number  wrong.  Parts;  1;  items;  60; 
working  time:  3  minutes. 


16.  Jumbled  Words  -  EST03A.  Judge  whether  or  not  words  could  be  made  by  mixing  the  letters  of  a  given  word. 
Sample  Items:  Given  word:  start 

I.  stare  Answers;  I  and  II  can't  be  made;  III  can  be  made. 

II.  stars 

III.  tarts  Score:  Number  right  minus  number  wrong.  Parts:  2;  items 

per  part;  25;  working  time;  4  minutes. 

^Adapted  with  permission  from  a  test  by  James  J.  McGrath. 

Adapted  with  permission  from  a  test  by  L..  L.  Thurstone. 
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17.  Letter  Problemg  -  ESI02A.  From  a  table  of  letter -problem  eolutione,  judge  whether  larger  given  problems  can 
be  solved  directly,  solved  with  transpositions,  or  are  unsolvable,  . 


Sample  Items;  I.  T  Z  U 
II.  U  V  Z 

III.  X  Z  X 


Table; 

T  Z 

=  u 

Y  Y  =  W 

Y  X  =  T 

Z  Y 

=  V 

T  V  •=  Z 

V  W  =  Y 

X  X 

=  z 

Z  U  =  W 

U  U  =  T 

Answers;  I  can  be  solved  directly,  11  can  be  solved  only  with  transpositions;  and  III  is  unsolvable. 

Score;  Number  right  minus  one-half  number  wrong.  Parts:  3;  items  per  part:  10;  working  time:  12  minutes. 

18.  Letter  Triangle  -  CSS02B.  Find  the  pattern  of  the  letters  arranged  systematically  within  a  triangle. 


Sample  Item; 


f 


A.  h 

B.  j 

C.  g 

D.  f 

E.  a 


Answer;  C  (g)  Score:  Number  right  minus  one-fourth 

number  wrong.  Parts;  1;  items;  12; 
working  time;  9  minutes. 


19.  Letter  "U"  -  ESU06A.  Check  all  the  words  in  lists  that  contain  the  letter  "u". 


Sample  Item  s: 


(  )  sense 

(  )  short 

(y)  juice 
(  )  special 


Score;  Number  correctly  placed  X's  minus  number  incorrectly  placed  X's. 
Parts:  2;  items  pe r  part:  200;  working  time:  2  minutes. 


20..  Marking- Speed  Test.  Make  as  many  X's  as  you  can  in  the  rows  of  squares  provided 
Sample  Item 


Score;  Number  of  complete  X's  made 
within  squares.  Parts;  1;  items:  180 
squares;  working  time:  1  minute. 


21.  Number  Classification 
given  numbers  each. 


Sample  Items: 


I.  44  55 

II.  10  45 


CSC03B.  Select  one  of  five  alternative  numbers  to  fit  into  each  of  four  classes  of  three 

Answers;  I,  D;  II,  E. 
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A.  421 

B.  53 

C.  219 

D.  22 

E.  25 


Score;  Number  right  minus  one-fourth  number  wrong. 
Parts;  1;  items:  16;  working  time;  5  minutes. 


22.  Number  Grouping  -  DSCOIB.  Group  given  numbers  into  several  different  classes  based  upon  properties  they  have 
in  common,  e.g.  ,  multiples  of  three. 

Sample  Item:  Given:  2  3  4  6  17  23  36  Possible  groups:  a.  2,  4,  6,  36  (even  numbers) 

b.  3,  17,  23  (odd  numbers) 

Score;  Number  of  groups  listed  that  exhaust  their  members.  Parts:  2;  items  per  part;  3;  working  time:  6  minutes. 

23.  Number -Group  Naming  -  CSC05A.  State  how  the  numbers  in  each  set  of  three  are  alike. 

Sample  Item:  35  110  75  Answer:  Divisible  by  5  (or  multiples  of  5) 

Score:  Number  of  correctly  named  groups.  Parts;  1;  items:  12;  working  time;  3  minutes. 

24.  Numerical  Operations*^  -  MSIOIB.  Part  III  of  the  Cuilford-Zimmerman  Aptitude  Survey  (adapted). 

Rapidly  add,  subtract,  dr  multiply  simple  numerical  problems  and  select  one  of  six  alternatives  as  the  answer. 

Score:  Number  right  minus  one-fifth  number  wrong.  Parts:  2;  items  per  part;  36;  working  time;  4  minutes. 

25.  Operations  Sequence  -  NSSOIB.  Produce  the  correct  order  of  three  specified  numerical  operations  in  order  to  get 
from  one  given  number  to  another. 

Sample  Item;  Start  with  6  a.  4  3  Answer:  b  a  c  Score:  Number  of  correct  orders  minus  one- 

obtain  18  b.  -f  2  fourth  number  of  incorrect  orders.  Parts:  2; 

c.  X  3  items  per  part:  12;  working  time:  10  minutes. 

26.  Perceptual  Speed*^  -  EFUOIA.  Part  IV  of  the  Guilford-Zimmerman  Aptitude  Survey  (adapted). 

Rapidly  match  each  of  five  objects  to  one  of  four  given  objects. 

Score;  Number  right  minus  one-fourth  number  wrong.  Parts:  1;  items:  72;  working  time:  5  minutes. 

Adapted  with  permission  from  Sheridan  Supply  Co. ,  Beverly  Hills,  Calif. 
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27.  Related  Words  I  -  ESR03A.  Choose  the  alternative  word  pair  with  a  relation  most  like  that  of  the  given  pair. 


Sample  Item:  GRAND  -  RAN 

A.  country  -  cot  Answer:  B  Score:  Number  right  minus  one -half  number  wrong. 

B.  respite  -  sit  Parts:  3;  items  per  part:  10;  working  time:  12  min- 

C.  loving  -  log  utes. 

28.  S  Test  -  ESI04A.  Discover  problems  in  items  composed  of  numbers,  letters,  and  words  and  solve  the  problems. 

Score:  Number  of  problems  solved  which  are  symbolic  in  nature.  Parts:  1;  Items:  20;  working  time:  5  minutes. 

29-  Seeing  Trends  II  -  CSROIB.  Describe  a  trend  based  upon  relations  of  letters  in  a  group  of  words. 

Sample  Item:  rated _ crate _ morning _ dearth _ separate  Answer:  The  "r"  moves  one  letter  to  the 

right  In  each  word. 

Score:  Number  of  correct  descriptions.  Parts:  1;  items:  15;  working  time:  10  minutes. 

30.  Series  Relations  -  ESS06A.  Choose  one  of  three  arithmetic  operations  that  best  relates  each  number  in  a  given 
series  to  the  previous  number. 

Sample  Item:  17,  9,  2  (given  series)  A.  -  8  Answer:  A  (none  is  correct,  but  A  is  best) 

B.  4  2 

C.  -  7 

Score:  Number  right  minus  one-half  number  wrong.  Parts:  2;  items  per  part:  15;  workihg  time:  8  minutes. 

31.  Sign  Changes  -  NSIOIA.  Solve  simple  arithmetic  problems  in  which  the  operation  sign  is  changed  according  to  a 
set  of  rules. 

Sample  Item:  Rules:  wherever  you  see:  -  replace  it  by  x  3-6=18  Score:  Number  right  solutions . 

+  replace  it  by  -  6  +  2  =  Parts:  1;  items:  32;  working 

4  -  3  =  17  time:  1  minute. 

32.  Sign  Changes  II  -  ESROIC.  Choose  the  sign  changes  that  make  the  expressions  into  equations. 

Sample  Item:  3  t  1  =  6x2  A.  Instead  of  +  you  -  Answer:  C 

B.  Instead  of  +  you  x 

C.  Instead  of  x  you  - 

D.  Both  A  and  C 

Score:  Number  right  minus  one-third  number  wrong.  Parts:  2;  items  per  part:  10;  working  time:  8  minutes. 

33.  Similar  Pairs  -  ESR04A.  Judge  whether  or  not  the  relation  in  the  second  pair  of  words  is  the  same  as  the  relation 
in  the  first  pair. 


Sample  Items: 

I. 

kire  -  lire 

fora  -  gora 

Answers:  I  and  II  liave  same  relation;  III  does  not. 

II. 

brake  -  rake 

freed  -  reed 

III. 

moan  -  noam 

toes  -  seot 

Score:  Number  right  minus  number  wrong.  Parts: 

2;  items  per  part:  15;  working  time:  6  minutes. 

34,  Sound  Grouping  - 

ESC04A.  Find  the 

word  that  does  not  belong  in  a  group  of  four  words. 

Sample  Item: 

A. 

comb 

Answer:  D 

Score:  Number  right  minus  one-third  number  wrong. 

B. 

foam 

Parts;  2;  items  per  part;  15;  working  time:  6  min¬ 

C. 

home 

utes. 

D. 

come 

35.  Symbol  Grouping  -  CSIOIB.  Rearrange  scrambled  symbols  in  a  specified  systematic  order  as  efficiently  as  pos¬ 
sible. 

Sample  Item:  Given:  x-ox-x  By  moving  one  or  more  adjacent  symbols  at  a  time,  arrive  at  the  order  xxx--o. 

Answer:  (the  minimum  number  of  moves  is  two) 

Score:  Two  points  for  the  most  efficient  rearrangement;  one  point  for  a  less  efficient  rearrangement;  and  no  points  for 
an  inefficient  or  incorrect  rearrangement.  Parts:  1;  items:  12;  working  time:  7  minutes. 

36.  Symbol  Identities  -  ESU07A.  Judge  whether  both  members  of  pairs  of  words  and  numbers  are  the  same  or  are 
different. 

Sample  Items:  2  16  3  ^  2  16  3  Score:  Number  right.  Parts:  2;  items  per  part:  50; 

Hahn,  Lorena  q  Hahn,  Lorina  working  time:  4  minutes. 

Bob  Ulm  O  Rob  Ulm 
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37.  Symbol  Manipulation  -  ESR02C.  Judge  whether  symbolic  conclusions  are  true  or  false  based  upon  given  premises. 


Sample  Items;  If  X  is  smaller  than  Y  (X  s  Y).  then:  I.  X  e  Y  (X  equals  Y) 

II.  X  nl  Y  (X  is  not  larger  than  Y) 

Answers:  I  and  III  are  false;  II  is  true.  III.  X  1  Y  (X  is  larger  than  Y) 


Score;  Number  right  minus  number  wrong.  Parts;  Z;  items  per  part;  15;  working  time;  8  minutes. 

38.  Symbol  Reasoning  -  ESI03A.  Decide  whether  symbolically  stated  conclusions  are  true,  false,  or  uncertain,  based 
upon  a  given  symbolic  relationship. 


Sample  Items: 

Given:  2x 

I. 

<  3y  <  Zz 

2x  =  Zz 

Answers; 

I.  false 

Score:  Number  right 

minus 

one -ha If  number  wrong. 

II. 

y  <  z 

II. 

true 

Parts:  2;  items  per 

part: 

24;  working  time:  12 

III. 

X  =  y 

III. 

uncertain 

minutes. 

39.  Typing  Errors  -  EST02A.  Choose  one  of  three  correctly  typed  words  that  the  given  incorrectly  typed  word  would 
most  likely  be.  Typing  rules  and  keyboard  are  printed  on  the  test. 


Sample  Item: 

F  H  E  E 

A. 

thee 

Answer:  A 

Score:  Number  right  minus,  one 

-half  number  wrong 

B. 

tree 

Parts:  2;  items  per  pant:  15; 

working  time;  10 

C. 

free 

minutes. 

40.  Varied  Symbols  -  DSC03B.  Find  the  different  common  properties  that  sets  of  letter  combinations  may  have  in 
common. 

Sample  item:  The  set|EPZT  |  APCTO  I  UMDT|is  like  which  of  these  groups: 


1.  ACBE  Possible  answers:  1  (start  with  vowels);  5  (have  three  consonants) 

2.  ROS 

3.  COM 

4.  GAIH  Score;  Number  of  indicated  groups  having  common  properties.  Parts;  2;  items 

5.  ZMOD  per  part;  5;  working  time:  8  minutes. 


41.  Way-Out  Numbers  -  ESS07A.  Choose  the  one  alternativte  number  from  a  list  of  four  ordered  numbers  that  is  far- 
thest  away  from  the  other  three. 


Sample  Item: 


A. 

B. 


31 

36 

45 

47 


Answer;  A 


Score;  Number  right  minus  number  wrong.  Parts; 
2;  items  per  part:  15;  working  time:  8  minutes. 


42.  Word  Changes  -  NSS02C.  Arrange  a  list  of  words,  each  containing  the  same  number  of  letters,  so  that  the  Hrst 
word  is  changed  into  the  last  word  with  only  one  letter  change  at  each  step. 


Sample  Item;  BELL  Answer;  2,  1,  3 

1.  BAIL 

2.  BALL  Score;  One  point  each  for  correct  order  of  first  five  Uems; 

-  3.  MAIL  two  points  each  for  correct  order  of  last  five  items.  Parts: 

1;  items;  10;  working  time;  6  minutes. 

43.  Word  Choice  -  ESC03A.  Choose  one  of  three  alternative  words  that  best  fits  the  given  class  of  words.  The  word 
classes  are  based  upon  common  properties  of  letters. 


Sample  Item: 


school 

fleet 

doomsday 


44,  Word  Combinations  -  CSU06A. 


A.  delete  Answer;  A  (none  are  correct,  but  A  is  best) 

B.  relate 

C.  expect  Score;  Number  correct  minus  one-half  number  wrong. 

Parts;  2;  items  per  part:  15;  working  time;  10  minutes. 

Produce  a  new  word  from  the  ending  of  one  word  and  the  beginning  of  another. 


Sample  Items: 

Given;  1. 

bridge 

A. 

duress 

Answers;  1. 

D  (gene) 

2. 

beam 

B. 

ze  ro 

2. 

C  (ample) 

3. 

open 

C. 

pledge 

3. 

A  (endure) 

D. 

need 

E. 

none  of  these 

Score;  Number  right  minus  one-fourth  number  wrong.  Parts;  2;  items  per  part:  15;  working  time:  8  minutes. 


45.  Word  Patterns  -  CSI03C.  Arrange  a  list  of  short  words  efficiently  in  a  crossword-puszle  design. 

Sample  Item:  Given  words;  bats,  easy,  hot,  tea,  the  Answer;  most  efficient  arrangement; 

Score;  Reciprocal  of  the  total  number  of  inefficient  and  incorrect  arrangements  of  letters  and  words 
multiplied  by  100,  Parts:  1;  items;  6;  working  time;  12  minutes. 


b 

tea 

hot 

easy 
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46.  Word  Relation!  -  CSR02B.  Recosnize  the  eame  relation  between  word!  in  each  o(  two  pair!,  then  complete  a  third 
pair  from  five  alternative  word!  ualng  the  eame  relation. 


Sample  Item: 

on  -  no 

A. 

art 

Answer; 

top  -  pot 

B. 

pat 

part  -  ? 

C. 

rapt 

D. 

tar 

E. 

trap 

E  Score:  Number  right  minui  one-fourth  number 

wrong.  Part!:  2;  item!  per  part:  10;  working 
time:  6  minutes. 


47.  Word  Transformation  -  NST02B.  Separate  letters  of  words  in  a  phrase  with  vertical  lines  to  make  a  different  set 
of  words. 


Sample  Item  (answer  marked): 


HE 


Score:  Number  of  vertical  lines  regrouping  all 
the  letters  in  the  given  phrase.  Parts:  1;  items: 
15;  working  time:  4  minutes. 


48.  Rating  of  Test-liking.  Rate  how  much  you  liked  the  last  test  on  a  five-point  scale.  The  five  categories  of  rating 
are:  very  much;  pretty  much;  a  little;  not  very  much;  not  at  all. 


Score:  Sum  of  standardized  ratings  for  all  47  tests. 

49.  Rating  of  Effort  Spent  on  Teat  Booklet.  Rate  how  hard  you  worked  on  this  booklet  on  a  five-point  scale.  The  five 
categories  of  rating  are:  as  hard  as  I  could;  very  hard;  fairly  hard;  not  very  hard;  not  hard  at  all. 

Score;  Sum  of  standardized  ratings  for  all  8  booklets. 

50.  Sex  Membership.  Score;  Boys  were  given  a  score  of  1;  girls  a  score  of  0. 


51.  Achievement-Prediction  Discrepancy  Score.  Score:  Four-year  high-school  grade-point  average  minus  the  least- 
squares  prediction  of  that  average  by  the  eight  subtests  of  the  Differential  Aptitudes  Test. 


52.  Iowa  Tests  of  Educ.itional  Development  -  Test  8,  General  Vocabulary  -  CMU.  Recognize  the  meanings  of  words 
commonly  used  in  communication.  This  test  is  similar  to  standard  verbal  comprehension  (CMU)  marker  tests. 

53.  Preliminary  Scholastic  Aptitude  Test  -  Verbal  -  CMU.  PSAT  is  an  abbreviated  form  of  the  SAT.  The  verbal  score 
is  the  sum  of  scores  on  four  tests;  Opposites,  Sentence  Completion,  Analogies,  and  Reading  Comprehension.  The 
dominant  saturation  is  hypothesized  to  be  CMU  with  some  CMR  variance  contributed  by  the  Analogies  test. 

54.  Cooperative  School  and  College  Ability  Test,  Form  2A  -  Verbal  -  CMU.  Verbal  score  is  composed  of  scores  on 
two  tests.  Sentence  Understanding  and  Word  Meanings.  Tests  similar  to  each  have  previously  loaded  on  the  verbal 
comprehension  (CMU)  factor. 

55.  Mathematics  Experience.  A  weighted  total  of  the  number  of  high-school  mathematics  courses  completed  and  cur- 
rently  being  taken.  One  point  was  given  for  each  semester  of  the  following  courses:  Consecutive  Mathematics  (grades 
5-8),  Remedial  Arithmetic,  Preparatory  Algebra  Arithmetic,  Algebra  I,  and  Geometry  I,  and  two  points  for  each  se¬ 
mester  of  Modern  Algebra,  Introductory  Algebra-Trigonometry,  Modern  Intermediate  Mathematics,  Modern  Geometry, 
and  College  Mathematics. 

56.  Typing  Experience.  Total  number  of  semesters  of  high-school  typing  courses  completed  and  currently  being  tak¬ 
en.  One  point  was  given  for  each  semester  of  the  following  courses:  Junior-High  Typing,  Typing  I,  and  Typing  II. 

57.  Shorthand  Experience.  Total  number  of  semesters  of  high-school  shorthand  courses  completed  and  currently  be¬ 
ing  taken.  One  point  was  given  for  each  semester  of  Shorthand  I  and  Shorthand  II. 

58.  Test  Administrator.  Score:  ISO  examinees  tested  by  one  examiner  were  given  scores  of  0;  75  tested  by  another 
examiner  were  given  scores  of  1. 
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This  study  approaches  the  problem  of  describing  symbolic  judgmental  pro¬ 
cesses  from  the  standpoint  of  individual  differences  in  terms  of  basic  traits.  Based 
upon  Guilford's  structure -of-intellect  model,  six  factors  of  symbolic  evaluation 
were  hypothesized  to  exist  as  distinct  from  each  other  and  also  from  factors  rep¬ 
resented  in  certain  other  domains  of  the  model.  (U) 

Twenty-five  experimental  tests  were  developed  to  serve  as  measures  of  the 
six  hypothesized  factors.  In  order  to  obtain  factors  clearly  generalizable  to  those 
generated  from  the  model,  at  least  three  tests  were  developed  for  each  hypothesized 
factor.  Furthermore,  the  tests  designed  were  not  specific  to  a  certain  kind  of 
symbol  but  utilized  several  kinds,  including  numbers,  words,  letters,  and  syl¬ 
lables.  (U) 

Measures  of  reference  factors  from  the  structure  of  intellect  were  also 
included  in  order  to  demonstrate  clearly  the  uniqueness  of  the  hypothesized  factors. 
Twenty-five  tests  designed  tO  measure  12  reference  factors  were  analyzed  as  part 
of  the  tota  1  batte ry.  In  addition,  three  variables  designed  to  measure  the  examinees' 
levels  of  motivation  to  succeed  on  the  aptitude  tests  were  employed  in  expectation 
of  isolating  a  motivation  factor  and  possibly  determining  the  influence  of  motivation 
upon  the  test  scores.  (U) 

All  the  tests  were  administered  to  ZZ5  high-school  seniors  in  a  Los  Angeles 
County  school.  The  correlation  matrix  was  factor  analyzed  and  axes  were  analyti¬ 
cally  rotated  to  a  least-squares  fit  to  a  simple -structure  target  matrix.  Twenty 
factors  were  isolated  and  named,  including  the  six  hypothesized  symbolic -evaluation 
factors ,  the  twelve  reference  factors,  and  two  different  factors  defined  by  the  moti¬ 
vation  measures.  (U) 


The  general  conclusion  of  this  research  is  that  the  structure-of-intellect 
model  has  continued  to  lead  fruitfully  to  undiscovered,  differentiable  intellectual 
abilities.  More  specifically,  five  new  factors  were  added  to  the  list  of  recognized 
intellectual  abilities  and  others  have  been  verified.  (U) 
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